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Abstract 


While  a  large  number  of  partnerships  form  as  defensive  measures  in  response  to  fierce 
global  competition,  distress  over  future  uncertainties,  and  a  lack  of  alternative  methods  to  ensure 
continued  survival,  synergistic  partnerships  are  characterized  as  being  cooperative  learning 
experiences  that  benefit  all  the  parties  involved.  The  best  partnerships  are  those  that  develop  into 
strategic  alliances  helping  to  capture  and  create  value  that  would  otherwise  have  been  difficult  to 
realize  if  not  for  the  mutually  shared  goals  and  resources  of  the  partnership.  In  this  paper,  we 
discuss  how  government,  industry,  and  academia  are  able  to  converge  upon  a  new  maintenance 
paradigm  aimed  at  benefiting  our  nation’s  military  forces.  In  particular,  representatives  from  all 
three  domains  are  working  together  to  determine  how  condition-based  maintenance  (CBM)  can 
best  serve  U.S.  Army  aviation  and  bolster  our  soldiers  engaged  in  the  war  against  terrorism. 
Described  as  is  a  set  of  maintenance  processes  and  capabilities  aimed  at  improving  U.S.  Army 
aviation  fleet’s  operational  readiness  and  reducing  soldiers’  maintenance  burden,  CBM  leverages 
advanced  technologies  to  help  generate  enhanced  diagnostics  for  key  components  on-board  a 
select  number  of  AH-64  Apache,  UH-60  Blackhawk,  and  CH-47  Chinook  helicopters.  The  near 
real-time  assessment  of  data  from  the  embedded  sensors  seeks  to  provide  the  U.S.  Army  with  a 
more  effective  and  efficient  way  to  conduct  maintenance  based  on  need  rather  than  scheduled 
periods,  the  capability  to  perform  supply  chain  actions  in  a  more  proactive  manner,  and  the 
ability  to  optimize  the  competing  demands  of  warfighting  and  planned  maintenance.  In  short, 
CBM  attempts  to  improve  the  way  the  U.S.  Army  approaches  maintenance,  transforming  it  from 
the  industrial  age  of  the  20th  Century  into  the  information  age  of  this  new  century.  We  believe 
that  through  the  successful  partnering  of  government,  industry,  and  academia,  we  will  be  able  to 
exemplify  how  CBM  is  demonstrating  business  transformation  for  the  U.S.  Army. 
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Chapter  1:  Introduction 


“ Partnership .  n.  A  relationship  between  individuals  or  groups  that  is 
characterized  by  mutual  cooperation  and  responsibility,  as  for  the  achievement 
of  a  specified  goal.  ” 

—American  Heritage  Dictionary 

Over  the  past  several  years,  the  Operations  Research  of  Excellence  (ORCEN)  in  the 
Department  of  Systems  Engineering  at  the  United  States  Military  Academy  (USMA)  has  worked 
with  the  U.S.  Army  Aviation  and  Missile  Command  (AMCOM)  on  research  projects  aimed  at 
enhancing  our  military  aviation  capabilities.  Previous  works  by  Major  Mark  Gorak  [1]  and 
Major  Stephen  Henderson  [2]  have  helped  to  lay  the  foundation  for  the  current  partnership 
between  government,  industry,  and  academia  that  aims  to  improve  the  U.S.  Army’s  helicopter 
maintenance  paradigm.  Under  the  direction  of  the  AMCOM,  support  of  the  Westar  Aerospace 
and  Defense  Group,  and  academic  research  focus  of  the  ORCEN,  we  continue  to  strengthen  our 
mutual  interest  in  determining  how  CBM  can  best  advance  our  military  into  the  information  age 
of  the  21st  Century. 

We  approached  this  year’s  joint  venture  as  a  mutually  beneficial  arrangement  that  would 
generate  not  only  tangible  results  for  our  partners,  but  also  relevant  and  significant  studies  for  us 
as  military  operations  research  faculty  members.  Being  new  to  the  partnership,  we  employed  the 
Systems  Engineering  and  Management  Process  (SEMP)  that  we  teach  to  cadets  at  USMA — a 
structured  problem  solving  process  useful  in  the  design  of  multidisciplinary,  large-scale,  and 
complex  engineering  problems  graphically  portrayed  in  Figure  1 — to  ensure  a  swift  and  prompt 
integration  of  appropriate  interests,  capabilities,  and  resources  [3],  While  the  application  of  the 
SEMP  can  vary  significantly  from  project  to  project,  reflecting  local  practices  and  expectations, 
we  believe  that  such  variance  often  provides  opportunities  to  adopt  new  ideas  and  new  ways  of 
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approaching  an  issue.  As  a  new  maintenance  paradigm  that  leverages  advanced  technologies  and 
introduces  new  processes,  CBM  is  an  ideal  project  for  the  application  of  the  SEMP. 


Descriptive 

Scenario 


Environment 


°rai/ 


Assessment  &  Feedback 


Normative 

Scenario 


Figure  1:  The  SEMP  Framework  [4] 


Chapter  2:  Using  the  SEMP  to  Foster  Mutual  Cooperation 

As  a  design  methodology  and  problem  solving  framework,  the  SEMP  provides  us  with  a 
coherent  and  structured  process  for  defining  and  agreeing  on  shared  goals  and  objectives, 
nominating  areas  for  research  focus,  and  selecting  appropriate  analytical  tools,  processes,  and 
methodologies  for  achieving  project  deliverables.  We  used  the  problem  definition  phase  of  the 
SEMP  to  ensure  our  unique  skills  and  analytical  capabilities  would  be  utilized  in  ways  that  best 
complements  our  partners’  expectations.  Much  of  this  initial  phase  was  spent  ironing  out  project 
deliverables  that  truly  satisfied  the  partners’  needs  and  requirements.  This  initial  phase  of  the 
project  also  helped  foster  mutual  cooperation  in  the  following  ways: 


•  Alleviated  much  of  the  anxiety  with  getting  the  initial  statement  of  work  (SOW)  produced, 
especially  in  terms  of  addressing  milestones,  timelines,  and  project  deliverables; 

•  Provided  a  deliberate  and  methodical  method  for  generating  common  understanding  of 
capabilities  and  limitations  among  partners; 

•  Prevented  partners  from  feeling  locked  into  a  set  of  pre-defined  objectives  and  goals, 
especially  for  projects  where  dynamic  exchange  is  to  be  expected; 

•  Empowered  ORCEN  analysts  with  the  capability  of  refining  the  initial  SOW  into  a  project 
contract  that  they  can  take  ownership  of  with  agreement  from  the  partners; 

•  Encouraged  partners  to  communicate  to  one  another  their  expectations  and  goals, 

•  Allowed  for  partners  to  come  up  with  a  SOW  based  on  system  needs,  common  interests,  and 
shared  responsibilities. 

Furthermore,  the  growth  of  such  mutual  cooperation  diminished  the  likelihood  that  the 
CBM  partnership  would  turn  into  a  short-lived  reactive  association  and  increased  the  chances  of 
it  developing  into  a  proactive  partnering,  and  perhaps  even  a  strategic  alliance.  Table  1  provides 
examples  and  characteristics  of  the  various  types  of  partnerships  that  exist  in  government, 
industry,  and  academia.  While  the  use  of  the  SEMP  does  not  guarantee  the  creation  of  longer- 
term  alliances,  it  does  help  to  communicate  common  expectations  and  shared  responsibilities 
that,  in  turn,  help  avert  purely  reactive  associations  from  forming. 

Table  1:  Partnerships  Types  and  Characteristics 

Government  Industry  Aeademin 

Reactive  Associations  Political  Expediency  Fierce  Competition  External  Locus  of  Control 

Military  Necessity  Fear  of  Uncertain  Future  Credentialing 

Proactive  Partnering  Balance  of  Core  Competencies  Focus  on  Core  Competencies  Internal  Locus  of  Control 

Longer-Term  Partnerships  Longer-Term  Mergers  Fruitful  Collaboration 

Strategic  Alliances  Shared  Resources  Common  Vision  Mutual  Respect 
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Chapter  3:  A  Roadmap  for  Getting  the  Most  out  of  a  Partnership 

Thriving  partnerships  and  successful  project  deliverables  are  predicated  upon  the  four 
key  dimensions  of  preparation,  promotion,  implementation,  and  documentation.  Figure  2 
illustrates  that  these  four  dimensions  are  not  mutually  exclusive  and  depicts  the  elements  that 
comprise  each  of  the  dimensions.  Central  to  overall  success  is  the  organizational  commitment  to 
the  program.  While  it  is  difficult  to  predict  how  committed  our  government  will  be  to  CBM  in 
the  future,  AMCOM’s  current  level  of  effort  and  focus  bodes  well  for  its  near-term  prospects. 
Not  only  has  AMCOM  provided  considerable  resources  in  the  promotion  of  CBM,  both  in  terms 
of  time  as  well  as  funding,  it  has  formulated  clear  objectives  and  goals  for  CBM.  As  a  result, 
AMCOM  has  helped  to  establish  a  common  vision  for  the  direction  of  CBM  and  has  brought 
industry  and  academia  together  to  work  on  preparing,  documenting,  and  implementing  a  viable 
strategy  for  how  to  best  incorporate  CBM  into  the  military. 
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With  the  substantial  weight  of  the  vast  resources  under  its  charge,  AMCOM  has  been  the 
key  governmental  agency  overseeing  the  direction  of  CBM.  AMCOM  has  taken  steps  to 
advance  partnerships  with  industry  and  academia  so  that  it  is  able  to  leverage  the  core 
competencies  distinctive  within  each  of  these  two  domains.  Industry  tends  to  possess  valued 
competencies  such  as  speed,  responsiveness,  efficiency,  practical  expertise,  and  entrepreneurial 
thinking.  Academia,  on  the  other  hand,  tends  to  maintain  valued  competencies  such  as  research 
capabilities,  technical  expertise,  and  advancement  of  new  ideas.  Therefore,  all  three  domains 
enter  the  CBM  partnership  with  the  understanding  that  each  is  able  to  focus  on  its  own  unique 
competencies — the  government  with  its  resources  and  vision,  industry  with  its  speed  and 
practicality,  and  academia  with  its  theories  and  strength  in  conducting  research. 

In  keeping  with  this  common  understanding  pertaining  to  the  CBM  partnership,  the 
ORCEN  agreed  to  undertake  project  deliverables  that  did  not  fall  on  the  critical  path  of  CBM 
advancement.  As  a  result,  the  ORCEN  was  able  to  conduct  research  that  ranged  from  answering 
specific  questions,  such  as  how  to  leverage  simulation  capabilities  into  the  CBM  paradigm  (see 
Exhibit  1 — Rapidly  Attaining  Low  Hanging  Fruit),  to  initiating  a  pilot  study  on  initial  CBM 
helicopter  data  (see  Exhibit  2 — Delivering  on  Unique  Skill  Sets).  Additionally,  the  ORCEN  was 
able  to  integrate  information  on  CBM  from  both  government  agencies  as  well  as  corporate 
participants  to  help  better  articulate  the  intended  vision  for  this  new  military  maintenance 
paradigm  (see  Exhibit  3 — Driving  Toward  Greater  Mutual  Understanding).  Lastly,  the  ORCEN 
was  able  to  collaborate  with  Westar  Corporation,  a  key  industry  partner,  on  developing  new  and 
more  meaningful  metrics  under  a  CBM  paradigm  (see  Exhibit  4 — Researching  Common  Areas 
of  Interest).  All  four  of  these  deliverables  represent  distinct  research  projects  the  ORCEN  has 
undertaken  to  help  contribute  to  the  advancement  of  CBM  as  an  Army  transformation  initiative. 
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Chapter  4:  The  Promising  Path  Forward  for  CBM 


Not  only  does  CBM  provide  an  entirely  different  way  of  conducting  maintenance  for  the  U.S. 
Army,  it  introduces  a  great  number  of  potential  research  partnerships  between  government, 
industry,  and  academia.  Areas  that  lend  themselves  directly  to  further  study  include  production 
operations  management  (e.g.,  supply  chains,  inventory  control,  lean  thinking,  and  just-in-time 
processes),  quality  control  (e.g.,  six  sigma,  5S,  root  cause  analysis,  and  failure  modes  effects 
analysis),  statistical  methodologies  (e.g.,  hypothesis  testing,  multivariate  analysis,  pareto 
analysis,  and  stochastic  analysis),  continual  process  improvements  (e.g.,  theory  of  constraints, 
value  stream  mapping,  tear-down  analysis,  value  engineering,  and  benchmarking),  and  problem 
solving  techniques  (e.g.,  design  of  experiments,  modeling  and  simulation,  data  mining,  and  data 
farming).  Under  the  watchful  direction  of  the  government,  speedy  and  efficient  application  of 
resources  from  industry,  and  varied  ideas  and  research  focus  of  academia,  CBM  provides  a  large 
wealth  of  possibilities  for  continued  proactive  partnerships.  Figure  3  illustrates  the  synergistic 
growth  of  knowledge  that  can  occur  as  a  result  of  the  successful  partnership  of  all  three  domains. 


Knowledge  Growth  through  CBM  Partnership 


Growth  and  Expansion  of 
Knowledge  on  CBM  through 
Partnership  of  Government, 
Industry,  and  Academia 


Figure  3:  Growth  and  Expansion  of  Knowledge  through  the  CBM  Partnership 
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A  positive  Matthew  Effect  is  likely  to  result  from  the  knowledge  gained  through  the  CBM 
partnership  [6],  As  more  collaborative  engagements  between  the  partners  develop,  the  more 
trust  builds  among  government,  industry,  and  academia.  As  this  trust  grows,  the  greater  the 
confidence  each  partner  will  have  in  each  other’s  capabilities.  Each  domain’s  individual 
contribution  will  help  to  add  not  only  to  the  existing  body  of  knowledge  pertaining  to  CBM,  but 
more  importantly,  will  help  to  catalyze  the  other  domains  into  pursuing  further  analysis.  As  a 
result,  new  systems  and  processes,  improved  diagnostics  and  prognostics,  and  enhanced  control 
mechanisms  and  maintenance  approaches  will  likely  result  from  the  study  of  CBM.  And  as  a 
consequence,  the  U.S.  Army  becomes  the  key  beneficiary  from  the  growth  and  expansion  of 
such  knowledge. 


Chapter  5:  Conclusions 

According  to  Foley,  “The  means  of  production  is  less  and  less  the  sweat  of  our  brow,  or 
the  leveraging  of  our  muscle  power  with  steam  or  water  or  electric  power,  or  mindless  repetition 
of  work  on  the  assembly  line.  Rather,  the  means  of  production  increasingly  is  the  leveraging  of 
our  intellectual  power  with  computers”  [7],  It  is  our  belief  that  our  partnership  on  CBM  is 
helping  to  leverage  computational  power  and  technological  advances  to  make  the  task  of 
maintenance  more  effective  and  more  efficient  for  our  military  forces.  We  also  believe  our 
partnership  on  CBM  is  helping  to  advance  the  tenets  of  Army  Transformation.  As  former  Army 
Chief  of  Staff  General  Eric  Shinseki  and  former  Army  Secretary  Thomas  White  have  stated: 
“Soldiers  on  point  for  the  nation  transforming  this,  the  most  respected  army  in  the  world,  into  a 
strategically  responsive  force  that  is  dominant  across  the  full  spectrum  of  operations  .  .  .  The 
Army’s  Vision  [consists  of]  People,  Readiness,  Transformation — and  our  efforts  to  change 
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quickly  into  a  more  responsive,  deployable,  agile,  versatile,  lethal,  survivable,  and  sustainable 
force”  [8],  The  introduction  of  the  CBM  paradigm  will  help  to  promote  military  transformation, 
and  the  partnership  of  government,  industry,  and  academia  is  helping  to  demonstrate  how  we  can 
leverage  each  domain’s  unique  skills  and  capabilities  towards  the  fulfillment  of  the  Army  Vision 
in  this  new  century. 
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Appendix  A:  List  of  Abbreviations 


A 

AH 

Attack  Helicopter 

AMCOM 

U.S.  Army  Aviation  and  Missile  Command 

C 

CBM 

Condition-Based  Maintenance 

CH 

Cargo  Helicopter 

O 

ORCEN 

Operations  Research  Center 

S 

SEMP 

Systems  Engineering  and  Management  Process 

SOW 

Statement  of  Work 

U 

UH 

Utility  Helicopter 

USMA 

United  States  Military  Academy 

#s 

5S 

Sort,  Set  In  Order,  Shine,  Simplify,  Sustain 

6a 

Six  Sigma 

*This  table  is  sorted 

alphabetically 
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How  to  Leverage  Computer  Simulation  in  Condition-Based  Maintenance: 
Capabilities  are  Determined  by  the  Availability  and  Accessibility  of  Data 


Executive  Summary 

Many  companies  are  discovering  that  they  can  leverage  computer  simulation  to  help 
improve  their  competitiveness  and  help  achieve  cost  savings.  Simulation  modeling  is  becoming 
an  important  enabling  technology  that  is  enhancing  numerous  functions  such  as  engineering  and 
operations,  logistics,  research  and  development,  risk  analysis,  process  design,  and  information 
systems.  Government  agencies  are  also  adopting  computer  simulation  techniques  to  assist  in 
areas  such  as  accident  analyses,  operator  training,  environmental  protection,  and  efficiency 
studies. 


In  the  past,  tailor-made  computer  simulators  were  developed  from  scratch  and  used 
almost  exclusively  by  mathematicians  and  programmers  who  held  the  expertise  to  utilize  and 
understand  such  time-consuming  and  expensive  systems  [15],  As  computers  have  gained  greater 
processing  power  at  more  affordable  prices,  however,  a  greater  number  of  users  are  taking 
advantage  of  simulation  capabilities.  As  a  result,  a  greater  number  of  companies  are  now  able  to 
leverage  computer  simulation  to  tackle  even  more  demanding  problems.  And  as  a  greater 
number  of  applications  are  being  developed,  computer  simulation  is  helping  to  provide  benefits 
such  as  [10,  15]: 

•  Higher  quality  manufacturing  through  reduced  costly  mistakes 

•  Reduced  risk  and  improved  safety 

•  Shorter  commissioning  time  of  new  products 

•  Increased  operational  capacity  and  usage 
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•  Savings  in  training  costs  of  personnel 

•  Extension  of  equipment  life 

•  Increased  confidence  in  major  decisions 

Since  these  benefits  all  appear  to  tie  directly  to  the  goals  of  condition-based  maintenance 
(CBM),  the  question  of  whether  or  not  computer  simulation  makes  sense  for  the  U.S.  Army 
aviation’s  new  maintenance  paradigm  ought  to  be  apparent  (refer  to  page  3  for  CBM  goals).  Not 
only  does  computer  simulation  have  the  potential  to  improve  many  functions  within  CBM,  it  also 
has  the  ability  to  help  promote  the  CBM  vision  of  achieving  optimal  operational  readiness  of  the 
aviation  fleet.  Consequently,  rather  than  focusing  on  trying  to  answer  whether  computer 
simulation  modeling  can  be  used  to  enhance  CBM,  it  is  perhaps  more  worthwhile  to  focus  on 
how  to  best  introduce  computer  simulation  modeling  into  the  CBM  process  in  the  most  effective 
manner.  This  paper  presents  three  major  alternatives  for  addressing  how  to  introduce  computer 
simulation  techniques  into  CBM: 

•  Develop  computer  simulation  tools  in-house  that  are  tailored  specifically  to  U.S. 
Army  Aviation  and  Missile  Command  (AMCOM)  engineering  requirements 

•  Use  existing  commercial  simulation  software  tools,  such  as  Crystal  Ball  and  Palisades 
Decision  Tools,  for  analysis  of  existing  data 

•  Outsource  the  modeling  functions  to  external  agencies  with  established  dependability, 
responsiveness,  and  expertise  in  computer  simulation 

Although  these  three  alternatives  are  very  distinct  courses  of  action,  there  should  be  no 
reason  to  think  that  each  one  cannot  play  a  role  in  advancing  the  CBM  paradigm.  It  is  very 
likely  that  the  best  decision  for  incorporating  simulation  into  CBM  will  include  elements  of  all 
three  alternatives.  But  regardless  of  the  decision,  it  is  important  to  recognize  that  the  success  of 
computer  simulation  techniques  in  CBM  depend  largely  on  the  availability  and  accessibility  of 
suitable  data. 

Introduction-- What  Are  Computer  Simulations  and  What  Can  We  Gain  from  Them? 

Computer  simulations  are  designed  to  provide  insights  into  the  dynamic  behavior  of 
systems  as  they  vary  over  time  [8,  16],  Traditionally,  the  formal  modeling  of  systems  has  been 
via  mathematical  models  that  attempt  to  find  analytical  solutions  to  problems  based  on  a  set  of 
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defined  parameters  and  initial  conditions.  Computer  simulations  are  often  used  as  adjuncts  to 
modeling  systems  in  which  analytical  solutions  are  not  possible  or  those  that  are  very  difficult  to 
answer.  There  are  many  different  types  of  computer  simulation,  but  the  common  feature  they  all 
share  is  their  attempt  to  generate  a  sample  of  representative  scenarios  for  a  model  in  which  a 
complete  accounting  of  all  possible  states  of  the  model  would  be  prohibitive  or  impossible  [9, 
13]. 


One  of  the  principal  advantages  with  computer  simulation  is  its  capacity  to  model  and 
represent  the  behavior  of  complex  systems  over  time  in  a  rapid,  convenient,  and  cost-effective 
manner  [14,  15].  Without  computer  simulations,  analysts  would  have  to  rely  upon  sophisticated 
mathematical  theories,  or  test  actual  devices  and  observe  how  they  behave  under  controlled 
conditions  to  learn  about  and  make  improvements  in  the  design  or  process  [4].  Another 
important  feature  of  computer  simulation  is  its  ability  to  emulate  every  significant  step  that 
occurs  in  a  process  and  identify  significant  interactions  between  resources  in  a  process.  This 
enables  analysts  to  gain  insights  about  the  impact  of  potential  decisions  or  changes  on  that 
process.  A  good  simulation  model  equips  the  analyst  with  a  visual  representation  of  what  will 
happen  in  the  process  if  changes  are  made  to  it,  and  it  gives  the  analyst  a  record  of  those 
changing  system  performance  measures  as  they  are  examined  under  various  scenarios  [13,  14], 

Simulation  has  been  used  most  extensively  in  situations  where  the  real  system  cannot  be 
used  for  experiments.  Computer  simulation  is  also  an  appropriate  analysis  tool  when  [3,  15]: 


•  The  real  system  does  not  yet  exist 

•  The  experiments  would  involve  high  economical  risks 

•  The  experiments  would  be  dangerous 

•  The  experiments  cannot  be  controlled  or  carried  out 

•  The  process  variables  are  too  difficult  to  be  measured  through  experimentation 

•  The  measurements  are  too  noisy 

•  The  experimenting  with  the  real  system  is  expensive 

•  The  system  cannot  be  easily  accessed 

•  The  dynamics  and  response  of  the  system  under  actual  conditions  are  too  slow 

•  The  proper  conditions  for  the  experiment  are  difficult  to  fulfill 

•  The  variables  of  the  system  cannot  be  easily  manipulated  in  an  experiment 
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Appendix  A  lists  both  a  wide-range  of  industry  sectors  employing  computer  simulation 
techniques  as  well  as  a  number  of  practical  applications  that  have  been  developed  through 
computer  simulation. 

Why  Does  Computer  Simulation  Make  Sense  for  CBM? 

Computerized  dynamic  simulation  models  are  useful  for  verification  of  both  conceptual 
and  detailed  process  designs  [15].  Because  they  make  in-house  pre-testing  of  automation 
systems,  user  interfaces,  and  operational  procedures  possible  in  a  relatively  quick,  efficient,  and 
economical  manner,  computer  simulation  appears  to  be  an  ideal  tool  for  enhancing  the  CBM 
paradigm.  According  to  DoD  Directive  5000.1,  the  stated  objectives  of  CBM  are  [5,  12]: 

•  Predict  equipment  failures  based  on  real-time  or  near  real-time  assessments  of 
equipment  condition  obtained  from  embedded  sensors 

•  Reduce  maintenance  down  time 

•  Increase  operational  readiness  by  repairing  or  replacing  system  components  based  on 
actual  condition  of  components  rather  than  on  a  scheduled  or  time-phased  basis 

As  such,  computer  simulation  can  help  to  facilitate  in  the  development  of  both  enhanced 
diagnostics  as  well  as  predictive  prognostics  which  serve  to  “improve  maintenance  agility  and 
responsiveness,  increase  operational  availability,  and  reduce  life  cycle  total  ownership  costs”  [7]. 
Because  the  effectiveness  of  computer  simulation  models  is  contingent  upon  large  amounts  of 
data,  the  profuse  amount  of  information  generated  from  the  embedded  sensors  across  the  aviation 
fleet  will  serve  to  provide  even  more  legitimacy  to  both  the  models  and  results  of  CBM  computer 
simulations.  Furthermore,  benefits  from  computer  simulation  need  not  be  limited  to  the 
operations,  support,  and  sustainment  phases  of  the  life  cycle  for  aviation  assets.  From  system 
acquisition  to  retirement  and  disposal,  computer  simulation  can  help  provide  robust  analysis  to 
nearly  all  phases  of  the  Total  Life  Cycle  Systems  Management— especially  when  faced  with 
decreased  testing  budgets,  more  complicated  systems,  more  software-intensive  systems,  more 
upgrades  to  existing  systems  as  part  of  “evolutionary  procurement,”  and  greater  interest  in 
system  reliability,  availability,  and  maintainability  [2], 
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What  Challenges  are  Associated  with  Incorporating  Computer  Simulation  into  CBM? 

There  are,  however,  numerous  costs  involved  with  the  introduction  of  computer 
simulation  modeling  that  we  need  to  be  cognizant  of.  These  costs  include  [1,  10,  16]: 

•  Computer  hardware— despite  steady  reduction  in  prices 

•  Simulation  software  testing,  development,  and  maintenance 

•  Software  training 

•  New  business  process  training 

•  Additional  labor  needed  to  attain  proficiency  in  new  software  and  business  processes 

•  Culture  change  implementation 

Although  the  introduction  of  computer  simulation  is  not  intended  to  replace  other  forms  of 
analysis,  it  may  initially  be  difficult  to  convince  analysts  to  immediately  employ  new  modeling 
techniques  that  they  may  not  be  familiar  with  and  have  relatively  little  confidence  in. 

It  is  also  important  to  consider  the  limitations  of  simulation  modeling.  Since  all 
simulation  models  contain  simplifications  and  assumptions  that  limit  their  accuracy,  computer 
models  attempting  to  capture  some  aspect  of  reality  must,  therefore,  be  imperfect  and  incomplete 
as  well  [9],  Consequently,  computer  simulations  will  not  eliminate  all  faults  or  failures  in  the 
aviation  fleet.  The  diagnostic  and  prognostic  capabilities  derived  from  simulation  modeling  are 
only  as  useful  as  the  data  from  which  the  models  originate.  Finally,  even  though  simulation  can 
help  to  reduce  defects  through  statistical  process  controls  such  as  Six-Sigma  [13,  16],  simulation 
will  not  fully  replace  operational  testing  [2],  Oftentimes,  simulation  works  best  as  a  modeling 
tool  that  helps  complement,  enhance,  and  substantiate  other  analytical  methods. 

How  Do  We  Best  Introduce  and  Leverage  Computer  Simulation  in  CBM? 

There  are  essentially  three  distinct  alternatives  we  can  consider  to  implement  computer 
simulation  within  CBM.  The  pros  and  cons  for  each  alternative  are  provided  below: 

•  Develop  computer  simulation  tools  in-house  that  are  tailored  specifically  to  U.S. 

Army  Aviation  and  Missile  Command  engineering  requirements  (Develop  In-House): 
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+  Simulation  tool  conforms  exactly  to  specified  requirements  and 
+  Developed  so  that  there  will  be  less  apparent  need  for  training  on  user  interfaces 

-  With  software  design  and  testing,  takes  arguably  the  longest  time  to  implement 

-  High  upfront  development  costs 

-  Unlikely  to  be  flexible  enough  to  exploit  unspecified  requirements 

-  Likelihood  of  product  obsolescence 

•  Use  existing  commercial  simulation  software  tools,  such  as  Crystal  Ball  and  Palisades 
Decision  Tools,  for  analysis  (Use  Existing  Commercial  Tools): 

+  Commercially  available  with  demonstrated  use  within  industry 
+  Available  software  support  and  expert  guidance 
+  Multiple  user  training  opportunities  available 
+  Relatively  inexpensive  compared  to  in-house  development 
+  High  likelihood  of  being  able  to  keep  up  with  product  upgrades 
+  Conformance  with  industry  and  academic  standards 

-  May  not  be  entirely  compatible  with  existing  data 

-  Models  may  not  be  congruent  with  existing  paradigms  and  analytical  frameworks 

-  Requires  operator  training 

-  May  not  be  proficient  enough  to  exploit  the  software  capabilities 

•  Outsource  the  modeling  functions  to  external  agencies  with  established  dependability, 
responsiveness,  and  expertise  in  computer  simulation  (Outsource  to  Experts): 

+  Able  to  leverage  existing  expertise  in  industry  or  academia  for  immediate  results 
+  Limited  long-term  software  development,  sustainment,  and  training  costs 
+  Possibility  of  forming  long-term  relationships  with  proven  partners 

-  In  the  long  run,  may  not  be  able  to  capture  and  reproduce  demonstrated  capabilities 

-  May  require  time-intensive  reverse  engineering  to  verify  conclusions 

-  Requires  a  certain  degree  of  trust  with  external  agencies 

-  Potential  to  become  reliant  on  proprietary  software 
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A  helpful  way  to  compare  these  three  potential  solutions  is  to  examine  the  pros  and  cons 
of  each  alternative  in  an  evaluation  matrix  format  [17],  as  shown  in  the  table  below: 


Table  1:  Evaluation  Matrix  to  Help  Decide  Best  Way  to  Leverage  Simulation  Capabilities 


Criteria: 

Ability  to  Tailor  Software  Specifications 

Develop  In- 

House 

+++ 

Alternatives: 

Use  Existing 

Commercial 

Tools 

0 

Outsource  to 

Experts 

0 

Cost 

... 

+++ 

+++ 

Time  to  Develop  Software 

... 

+++ 

+++ 

Amount  of  Required  Training  &  Education 

... 

+++ 

0 

Time  to  Implement  Capabilities 

... 

+++ 

+++ 

Compatibility  with  Existing  Data 

+++ 

... 

0 

Congruence  with  Existing  Paradigms 

+++ 

... 

0 

Verification  of  Simulation  Results 

+++ 

+++ 

... 

Flexibility  to  Adapt  to  Unknown  Requirements 

... 

+++ 

0 

Likelihood  of  Product  Obsolescence 

... 

+++ 

+++ 

Reliance  on  Long-Term  Support  Relationships 

0 

+++ 

... 

Legend: 

+++=Positive 

0=Neutral 

— =Negative 

Clearly  all  three  alternatives  have  their  advantages  and  disadvantages,  and  it  is  highly 
unlikely  that  a  single  alternative  will  provide  the  best  way  to  implement  computer  simulation  for 
CBM.  What  this  chart  does  provide,  however,  is  a  way  to  determine  how  to  gradually  introduce 
computer  simulation  to  different  portions  of  the  CBM  paradigm.  As  such,  the  best  solution  is 
probably  to  simply  choose  a  particular  alternative  for  a  portion  of  the  CBM  process  that  satisfies 
its  specific  characteristics,  needs,  and  requirements.  One  advantage  that  such  a  gradual  approach 
provides  to  CBM  is  the  ability  to  incrementally  learn  from  what  works  and  what  doesn’t. 
Another  advantage  to  this  diversified  approach  is  that  as  more  work  is  conducted  via  computer 
simulation,  analysts  will  increasingly  learn  more  about  the  capabilities  and  limitations  of 
simulation.  As  a  result,  they  will  be  able  to  become  both  better  consumers  of  simulation  tools  as 
well  as  better  advocates  for  the  future  use  and  implementation  of  simulation  capabilities. 
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Conclusion — What  Conditions  Make  it  Possible  for  CBM  to  Exploit  Computer  Simulation? 


Computer  simulation  is  a  very  powerful  tool  that  can  be  successfully  applied  to  nearly  all 
areas  of  CBM  and  nearly  all  stages  of  Total  Life  Cycle  Systems  Management.  It  can  help  to 
save  considerable  time  and  significant  costs  in  the  development  of  effective  diagnostics  and 
prognostics  for  improving  the  overall  readiness  of  the  U.S.  Army  aviation  fleet  while 
simultaneously  reducing  the  maintenance  man-hours  currently  imposed  on  our  soldiers. 
However,  it  is  the  accessibility  and  availability  of  suitable  data  that  will  determine  what 
computer  simulation  capabilities  will  be  used  to  help  enhance  the  CBM  paradigm  [10].  Limited 
data  that  is  cumbersome  to  retrieve  severely  hampers  the  value  of  not  just  computer  simulation, 
but  just  about  all  analytical  techniques.  Under  the  current  aviation  maintenance  paradigm,  only  a 
small  percentage  of  the  existing  data  is  being  exploited  to  better  improve  fault  detection  and 
optimize  the  operational  readiness  of  our  aviation  fleet.  Accordingly,  it  is  not  hard  to  image  that 
a  disproportionately  small  portion  of  our  analysts’  time  is  being  used  for  actual  analysis  while 
the  bulk  remainder  of  their  time  is  spent  simply  trying  to  gain  access  to  the  appropriate  data  and 
data  sources.  A  data  architecture  that  can  help  to  remedy  this  shortcoming  will  serve  as  a  key 
development  in  providing  CBM  with  its  enhanced  analytical  promise. 

It  will,  however,  take  time  to  train  and  educate  analysts  on  the  robust  capabilities  of 
computer  simulation.  Yet,  once  the  CBM  architecture  begins  to  materialize,  computer 
simulation  will  almost  assuredly  play  a  vital  role  in  the  program.  Furthermore,  a  positive 
Matthew  Effect  [11]  is  likely  to  develop  between  simulation  capabilities  and  CBM  outcomes. 
As  more  valid  data  becomes  readily  available  and  accessible,  the  more  confidence  there  will  be 
in  the  construction  of  the  simulation  models  that  draw  upon  the  data;  the  better  the  models 
become,  the  more  certainty  there  will  be  in  the  simulation  results;  the  better  the  simulation 
results  become,  the  more  trust  there  will  be  in  the  recommendations  inferred  from  those  results; 
the  better  the  conclusions  that  can  be  applied  to  the  aviation  fleet,  the  better  the  CBM  program 
will  become;  and  the  better  the  program  becomes,  the  more  CBM  can  demonstrate  that  is  able  to 
improve  aviation  maintenance,  increase  warfighting  readiness,  and  save  taxpayer  dollars. 
Consequently,  it  is  not  too  difficult  to  image  how  a  virtuous  cycle  begins  to  cultivate  among 
simulation  capabilities  and  the  CBM  paradigm  for  U.S.  Army  aviation  maintenance. 
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Exhibit  1,  Appendix  A: 


Table  1.  Industrial  Uses  and  Applications  of  Computer  Simulations 


The  Broad  Field  of  Computer  Simulation  Uses 

Industry  Sectors 

Simulation  Applications 

Aircraft 

Advanced  Control  of  Manufacturing 

Avionics 

Product  Simulation 

Biomedical 

Rapid  Prototyping 

Chemicals 

Efficiency  Studies 

Construction,  Civil 

Waste  Minimization 

Financial 

Business  Process  Models 

Flight  Training 

Financial  Analysis 

Food  &  Beverage 

Ordering  Policies  for  Inventory  System 

Hospitals 

Human  Aspects,  Ergonomics 

Insurance 

Environmental  Protection 

Machine  Tools 

Life  Cycle  Analysis  and  Prediction 

Mechanical  Engineering 

Accident  Analysis 

Medical 

Process  Design  and  Engineering 

Metals  Processing 

Logistics 

Military 

Research  and  Development 

Minerals 

Risk  Analysis  and  Risk  Mitigation 

Oil  &  Gas  Exploration 

Software  Testing 

Paper  &  Pulp 

Training  of  Users  /  Operators 

Pharmaceuticals 

Reengineering  of  Business  Processes 

Power  Generation 

Service  Organization  Design 

Rubber  &  Plastics 

Communication  Network  Protocols 

Ship  Building 

Space  Exploration 

Transportation 

Utilities 

Table  derived  from  SimServ  Website:  http://www.sim-serv.com/simulation.php.  Additional 
data  taken  from  Law,  A.,  and  W.  Kelton.  (2000).  Simulation  Modeling  and  Analysis,  3ld  ed. 
Boston:  McGraw  Hill. 
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ABSTRACT 


This  thesis  provides  a  stochastic  modeling  tool  to  assist  in  the  component  selection 
process  for  Army  Aviation’s  Condition-Based  Maintenance  Plus  (CBM+)  program.  The  CBM+ 
program  uses  measurements  from  sensors  to  monitor  the  health  (resistance  to  failure)  of 
components  replacing  the  Legacy  process  of  scheduled  inspections  and  maintenance.  This  work 
is  in  conjunction  with  the  Operations  Research  Center  of  Excellence  (ORCEN)  at  the  United 
States  Military  Academy  to  assist  in  providing  insight  for  the  U.S.  Aviation  and  Missile 
Command  (AMCOM).  AMCOM  is  currently  developing  CBM+  from  its  current  early  stages 
into  a  program  that  will  revolutionize  maintenance  procedures  for  the  Army’s  helicopter  fleet. 

The  subsystem  selected  for  this  thesis  is  the  AH-64/UH-60  T701C  Turbine  Helicopter 
Engine.  The  times  to  occurrences  to  serious  diagnostics  symptoms  requiring  maintenance  action 
were  collected  from  the  Army’s  Aviation  and  Missile  Research,  Development,  and  Engineering 
Center  (AMRDEC).  Data  analysis  indicates  that  a  nonhomogeneous  Poisson  process 
appropriately  models  the  times  between  the  occurrences  of  serious  diagnostic  symptoms  for  this 
engine.  A  Microsoft  Excel  simulation  utilizing  Crystal  Ball  version  5.5  compares  an  engine 
monitored  by  CBM+  versus  the  traditional  Legacy  system  of  maintenance.  This  simulation 
provides  information  on  diagnosed  faults,  mission  aborts,  repair  times,  false  positives,  and 
logistical  implications. 

This  simulation  is  generic  and  can  be  used  in  comparing  CBM+  candidate  components 
for  future  inclusion  into  the  CBM+  program.  Results  suggest  that  the  sensor’s  false  alarm  rate 
and  the  reliability/maintainability  of  the  CBM+  sensor  are  important  factors  to  consider.  Since 
the  CBM+  sensor  is  continually  monitoring  a  component’s  condition,  a  modest  sensor 
probability  of  detection  of  an  impending  failure  can  result  in  fewer  mission  aborts  than  those  that 
arise  in  the  Legacy  system.  The  ability  and  speed  of  the  logistics  system  to  recognize  and 
respond  to  sensor  measurements  indicating  impending  failure  affect  the  potential  success  of 
CBM+. 
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EXECUTIVE  SUMMARY 


U.S.  Aviation  and  Missile  Command  (AMCOM)  has  initiated  a  program  to  monitor 
Army  helicopter  component  health  (resistance  to  failure)  through  a  network  of  sensors  installed 
on  aircraft.  This  sensor  network  forms  the  foundation  for  a  maintenance  philosophy  known  as 
Condition-Based  Maintenance  Plus  (CBM+).  This  thesis  provides  a  stochastic  modeling  tool  to 
assist  in  the  component  selection  process  for  Army  Aviation’s  CBM+  program.  This  work  is  in 
conjunction  with  the  Operations  Research  Center  of  Excellence  (ORCEN)  at  the  United  States 
Military  Academy  to  assist  in  providing  insight  for  the  U.S.  Aviation  and  Missile  Command 
(AMCOM).  AMCOM  is  currently  developing  CBM+  from  its  current  early  stages  into  a 
program  that  will  revolutionize  maintenance  procedures  for  the  Army’s  helicopter  fleet.  The 
CBM+  program  uses  measurements  from  sensors  to  monitor  the  health  of  components  replacing 
the  Legacy  process  of  scheduled  inspections  and  maintenance.  Furthermore,  Army  Aviation’s 
CBM+  program  is  a  collection  of  maintenance  processes  and  capabilities  derived,  in  large  part, 
from  real-time  assessment  of  weapon  system  condition,  obtained  from  embedded  sensors  and/or 
external  tests  and  measurements.  Currently,  aircraft  parts  are  replaced  based  on  results  of 
scheduled  maintenance  inspections  of  the  Legacy  maintenance  system.  Under  CBM+,  the 
condition  of  components  will  be  monitored  and  the  components  replaced  when  sensors  show 
indications  of  possible  failure  and  extended  wear. 

Not  all  components  will  benefit  from  being  part  of  the  CBM+  program.  Stochastic 
modeling  and  simulation  are  used  to  develop  a  tool  to  assist  in  the  component  selection  process 
for  CBM+  program.  This  work  is  in  cooperation  with  the  Operations  Research  Center  of 
Excellence  (ORCEN)  at  the  United  States  Military  Academy  supporting  AMCOM. 

The  subsystem  selected  for  this  thesis  is  the  AH-64/UH-60  T701C  Turbine  Helicopter 
Engine.  The  times  of  occurrences  of  serious  diagnostics  symptoms  requiring  maintenance  action 
were  collected  from  the  Army’s  Aviation  and  Missile  Research,  Development,  and  Engineering 
Center  (AMRDEC)  and  analyzed  to  provide  input  to  the  development  of  a  simulation  model. 
The  resulting  Microsoft  Excel  simulation  utilizing  Crystal  Ball  version  5.5  compares  an  engine 
monitored  by  CBM+  versus  the  traditional  Legacy  system  of  maintenance.  The  output  of  the 
simulation  provides  information  on  diagnosed  faults,  mission  aborts,  repair  times,  false  positives, 
and  logistical  implications. 
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This  simulation  is  generic  and  can  be  used  in  comparing  CBM+  candidate  components 
for  future  inclusion  into  the  CBM+  program.  Results  from  the  simulation  study  suggest  that 
since  a  sensor  is  continually  monitoring  the  component,  it  doesn’t  have  to  be  highly  accurate  in 
diagnosing  impending  failures  in  order  to  produce  fewer  mission  aborts  than  the  Legacy 
maintenance  system.  However,  it  is  extremely  important  to  minimize  the  number  of  false 
positives  when  using  CBM+  sensor  otherwise  the  Legacy  maintenance  system  outperforms 
CBM+  with  respect  to  total  inspection/repair  time. 
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I.  INTRODUCTION 


A.  CONDITION-BASED  MAINTENANCE  PLUS 

U.S.  Aviation  and  Missile  Command  (AMCOM)  is  interested  in  monitoring  Army 
helicopter  component  health  through  a  system  of  sensors  and  monitors  placed  onboard  aircraft. 
Component  health  is  defined  as  a  component’s  ability  to  provide  the  proper  mechanical  action  it 
was  designed  and  engineered  to  accomplish.  This  program  is  in  response  to  a  Department  of 
Defense  (DoD)  strategy  which  states  that  all  services  should  seek  “operational  supportability”  in 
system  development  and  demonstration.  (DoD  5000.2,  2003)  Over  the  past  year,  approximately 
twenty  different  aircraft  components  were  monitored  by  sensors;  the  components  are  installed  on 
different  types  of  airframes.  The  AH-64  Apache,  UH-60  Blackhawk,  and  CH-47  Chinook  were 
selected  since  these  are  the  aircraft  that  are  included  in  the  Army’s  force  modernization  plan. 
This  sensor  network  forms  the  foundation  for  a  maintenance  philosophy  known  as  Condition- 
Based  Maintenance  (CBM). 

According  to  Defense  Acquisition  Guidebook  dated  20  December  2004  “the  goal  of 
CBM  is  to  perform  maintenance  only  upon  evidence  of  need.  CBM  tenets  include:  designing 
systems  that  require  minimum  maintenance;  need-driven  maintenance;  appropriate  use  of 
embedded  diagnostics  and  prognostics  through  the  application  of  Reliability-Centered 
Maintenance  (RCM);  improved  maintenance  analytical  and  production  technologies;  automated 
maintenance  information  generation;  trend  based  reliability  and  process  improvements; 
integrated  information  systems  providing  logistics  system  response  based  on  equipment 
maintenance  condition;  and  smaller  maintenance  and  logistics  footprints.”  (Department  of 
Defense,  2003)  A  more  specific  form  of  CBM  exists  and  that  is  known  as  Condition-Based 
Maintenance  Plus  (CBM+).  CBM+  “expands  on  these  basic  concepts,  encompassing  other 
technologies,  processes,  and  procedures  that  enable  improved  maintenance  and  logistics 
practices.  CBM+  can  be  defined  as  a  set  of  maintenance  processes  and  capabilities  derived,  in 
large  part,  from  real-time  assessment  of  weapon  system  condition,  obtained  from  embedded 
sensors  and/or  external  tests  and  measurements.  The  design  specifications  should  identify  early 
teaming  with  systems  engineering  to  clearly  define  and  understand  the  operating  envelope  in 
order  to  design  in  Built-In-Test  (BIT)  and  Built-In-Self-Test  (BIST)  mechanisms  including  false 
alarm  mitigation.”  (Department  of  Defense,  2006)  False  alarm  mitigation  is  accomplished  by 
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using  sensor  equipment  that  provides  for  enhanced  capability  for  fault  detection,  isolation,  and 
repair  time  minimization.  The  purpose  of  this  enterprise  is  to  provide  cost-effective  warning  of 
potential  catastrophic  failure  or  mission  abort. 

AMCOM’s  mission  statement  is  “to  transform  Army  Aviation  maintenance  to  Condition- 
Based  Maintenance,  by  converting  condition  and  usage  data  into  maintenance  actions”. 
(AMCOM  Web  Site,  2006)  Currently,  aircraft  parts  are  replaced  based  on  a  system  of  scheduled 
maintenance  inspections.  CBM+  would  drastically  change  this  system.  Under  CBM+, 
components  would  now  be  monitored  and  replaced  only  when  sensors  show  indications  of  ill 
health  or  possible  imminent  failure  and  extended  wear.  AMCOM  contacted  the  Operations 
Research  Center  of  Excellence  (ORCEN)  at  the  United  States  Military  Academy  to  assist  in 
providing  additional  insight  into  this  important  area  of  research.  This  thesis  is  produced  in 
conjunction  with  ORCEN  in  an  effort  to  provide  AMCOM  with  a  stochastic  model  and 
simulation  that  will  be  useful  in  aiding  decisions  concerning  the  potential  introduction  of  aircraft 
components  to  CBM+. 

Currently,  only  passive  sensors  are  employed  on  Army  aircraft.  After  each  flight  the 
crew  chief  downloads  data  from  the  sensors  onto  a  recording  device.  The  crew  chief  then 
transfers  the  data  to  a  laptop  computer  for  that  specific  aircraft.  All  of  the  crew  chiefs’  laptop 
computers  feed  into  a  desktop  computer  at  the  unit’s  Production  Control  office.  The  Production 
Control  office  is  the  controlling  node  for  all  maintenance  activities  of  an  Army  Aviation  unit. 
B.F.  Goodrich  contact  teams  are  at  specific  sites  and  are  assisting  these  Production  Control 
offices  with  interpreting  the  data.  Based  on  these  interpretations  these  data  can  trigger  repair  or 
replacement  of  components.  The  data  are  then  sent  from  the  Production  Control  office  to  a  data 
warehouse  which  is  currently  undergoing  construction  by  the  Westar  Corporation.  From  this 
data  warehouse  AMCOM  reviews  the  data  and  monitors  the  development  of  CBM+. 

B.  OBJECTIVES 

The  objective  of  this  thesis  is  to  utilize  stochastic  modeling  and  simulation  to  aid  in 
determining  which  aircraft  components  should  be  included  in  CBM+  Microsoft  Excel  2003  and 
Crystal  Ball  version  5.5  were  chosen  as  the  software  package  for  the  simulation  of  this  model. 
(Ragsdale,  2004)  Crystal  Ball  version  5.5  is  an  Excel-based  Monte  Carlo  simulation  application 
produced  by  Decisioneering  Incorporated.  The  components  that  have  been  selected  to  use 
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CBM+  were  chosen  as  a  test  bed  to  make  sure  that  the  CBM  process  results  in  a  decrease  of  the 
maintenance  burden  on  the  soldier,  an  increase  in  platform  availability  and  readiness,  and  a 
reduction  of  operations  and  support  costs.  (Brown,  2005)  Since  AMCOM  is  in  the  earliest 
phases  of  CBM+,  it  acquired  sensors  to  track  and  monitor  aspects  that  its  engineers  hypothesized 
would  have  a  useful  probability  of  providing  the  most  beneficial  results  in  terms  of  improving 
operational  readiness  and  reduction  of  maintenance  related  costs.  Now  that  results  have  been 
generated  there  needs  to  be  a  component  selection  system  established  that  will  be  used  when 
CBM+  is  implemented  across  the  Army  aviation  fleet. 

The  simulation  used  in  this  thesis  compares  a  CBM+  monitored  airframe  to  a  non-CBM+ 
monitored  airframe.  The  measures  of  performance  include  mitigation  of  mission  aborts,  time 
spent  repairing  components,  and  time  spent  awaiting  replacement  component  arrival.  Through 
this  comparison  it  is  possible  to  determine  which  components  to  enter  into  the  program  based  on 
the  greatest  reduction  of  mission  aborts  and  possible  gain  in  mission  performance  and 
operational  readiness.  This  can  be  a  useful  tool  for  AMCOM  to  use  in  future  decision  making. 

C.  LEGACY  AND  CONDITION-BASED  MAINTENANCE  PLUS  REGIMES 

A  Legacy  unit  that  conducts  maintenance  using  the  conventional  maintenance  regime  is 
currently  in  place  at  all  Army  aviation  units.  A  CBM+  unit  conducts  the  Legacy  maintenance 
regime  but  also  has  the  benefit  of  CBM+  sensors  installed  on  specified  components.  Both 
regimes  employ  intensive  scheduled  maintenance  and  reactive  unscheduled  maintenance.  The 
use  of  CBM+  monitoring  promises  to  reduce  numerous  time-intensive  scheduled  maintenance 
actions,  reduce  the  unexpected  nature  of  unscheduled  maintenance,  and  increase  operational 
readiness.  (Department  of  the  Army,  2004)  An  example  of  this  is  Air  Worthiness  Release 
(AWR)  dated  16  June  2005,  which  deleted  mandatory  inspection  requirements  for  six  different 
CBM+  monitored  components  on  AH-64  Apache  and  UH-60  Blackhawk  helicopters.  This 
AWR  saved  Maintenance  Man  Hours  (MMH)  per  inspection,  downtime  per  aircraft,  and  Time 
Between  Overhaul  (TBO)  for  components.  This  AWR  is  the  first  of  many  of  its  kind  that  will 
transition  Army  Aviation  from  the  present  rigid,  time-intensive,  and  reactive  Legacy 
maintenance  regime  to  a  prediction-based  CBM+  maintenance  regime. 
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D.  METHODOLOGY 


This  thesis  is  divided  into  six  chapters  and  follows  this  structure.  Chapter  One, 
“Introduction”,  describes  the  background,  objectives,  and  methodology  of  this  work.  Chapter 
Two,  “Aircraft  System  Structure  and  Data”,  discusses  in  detail  the  aircraft  components  that  are 
currently  monitored  by  Condition-Based  Maintenance  Plus  and  the  available  data  describing 
their  performance,  failure,  and  repair.  This  chapter  describes  the  maintenance  procedures  used 
by  Army  aviation  units  and  the  repair  times  associated  with  these  actions.  Chapter  Three, 
“Stochastic  Models  for  Comparing  Legacy  Maintenance  and  Condition-Based  Maintenance 
Plus”,  details  the  model  structures,  and  gives  example  cases  of  the  stochastic  model  that  is  used 
in  this  thesis.  Chapter  Four,  “Legacy  Maintenance  and  Condition-Based  Maintenance  Plus 
Simulation”,  describes  the  architecture,  characteristics,  assumptions,  and  results  of  the  simulation 
used  in  this  thesis.  Chapter  Five,  “Data  Analysis”,  presents  the  results  of  the  analysis  of  the 
simulation  output  as  different  components  are  monitored  and  compares  average  operational 
readiness  and  repair  times.  Finally,  Chapter  Six,  “Conclusions  and  Recommendations”, 
summarizes  the  findings  of  this  thesis  and  possible  uses  for  this  thesis  in  future  decision  making. 
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II.  AIRCRAFT  SYSTEM  STRUCTURE  AND  DATA 


A.  GENERAU  DESCRIPTION  OF  AIRCRAFT  TYPES  AND  COMPONENTS 

This  chapter  will  discuss  in  detail  the  aircraft  components  that  are  currently  in  the 
Condition-Based  Maintenance  Plus  program  or  are  under  consideration  for  inclusion.  Any 
available  data  describing  their  performance,  failure,  and  repair  are  listed.  Furthermore,  this 
chapter  describes  the  maintenance  procedures  used  by  Army  aviation  units  and  the  costs 
associated  with  these  actions;  all  CBM+  sensors  currently  use  vibratory  information  to  monitor 
these  components.  The  three  different  types  of  aircraft  presently  monitored  by  CBM+  are  the 
AH-64  Apache,  UH-60  Blackhawk,  and  the  CH-47  Chinook. 

1.  AH-64  Apache  Helicopter  Information 

The  AH-64  Apache  helicopter  is  a  twin-engine,  tandem-seat,  aerial  weapons  platform. 
(TM  1-1520-251-10,  2002)  Its  primary  mission  is  to  provide  attack  and  reconnaissance 
capabilities  in  support  of  the  ground  tactical  plan.  There  are  currently  thirteen  components  of 
eight  different  types  monitored  by  CBM+  on  the  AH-64.  A  listing  of  AH-64  Apache  CBM+ 
components  is  listed  in  Table  1 . 

Table  1 .  AH-64  Apache  CBM+  Component  Listing 


AH-64  COMPONENT 

NOMENCLATURE 

NSN 

PART 

NUMBER 

FEDLOG  NOMENCLATURE 

APU  Clutch 

3010-01-515-8483 

3617950-1 

CLUTCH  AS  SEMBLY,FRICTION 

Engine  Assembly,  70 1C 

2840-01-284-4011 

6071T24G01 

ENGINE,AIRCRAFT,TURBO-SHAFT 

Utility  Hydraulic  Pump 

4320-01-158-0893 

7-311810022-3 

PUMP.AXIAL  PISTONS 

Forward  Hanger  Bearing 

3130-01-333-8491 

7-311350008-5 

BEARING  UNIT.BALL 

Aft  Hanger  Bearing 

3130-01-333-8490 

7-211350007-5 

BEARING  UNIT.BALL 

MR  Pitch  Housing 

1615-01-235-5845 

7-311411215-13 

HOUSINGASSEMBLY 

MR  Upper  Mast  Bearing 

3110-01-215-4794 

7-311411202-5 

BEARING.ROLLER,  TAPERED 

MR  Lo 

wer  Mast  Bearing 

3110-01-179-7335 

7-114110011 

BEARING.ROLLER,  TAPERED 

Depicted  below  in  Figure  1  is  the  location  of  these  components  on  the  AH-64  Apache 
extracted  from  Figure  2-2  of  TM  1-1520-251-10.  The  number  in  parenthesis  indicates  the  total 
number  of  that  type  of  component  on  the  aircraft. 
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Figure  1.  Location  of  AH-64  Apache  CBM+  Components 


2.  UH-60  Blackhawk  Helicopter  Information 

The  UH-60  Blackhawk  helicopter  is  a  twin-turbine  engine,  single-rotor,  semimonocoque 
fuselage  helicopter.  Its  primary  mission  is  the  tactical  transport  of  troops,  supplies  and 
equipment.  Its  secondary  missions  include  training,  mobilization,  development  of  new  and 
improved  concepts,  and  support  of  disaster  relief.  (TM  1-1520-237-10,  2003)  There  are 
currently  nineteen  components  of  eight  different  types  monitored  by  CBM+  on  the  UH-60.  A 
listing  of  UH-60  Blackhawk  CBM+  components  is  listed  in  Table  2. 

Table  2.  UH-60  Blackhawk  CBM+  Component  Listing 


UH-60  COMPONENT 

NOMENCLATURE 

NSN 

PART  NUMBER 

FEDLOG  NOMENCLATURE 

Oil  Cooler  Fan  Bearing 

3110-01-329-8573 

1 10KSZZ-401 

BEARING.BALL,  ANNULAR 

Main  Rotor  Blade 

1615-01-106-1903 

70150-09100-043 

BLADE,MAIN  ROTOR 

Pump  Module  Assembly 

4320-01-207-7228 

70652-02300-050 

MODULE  ASSY.PUMP 

Damper  Assembly 

1615-01-285-3024 

70106-08100-046 

DAMPENER.FLUTTER 

Engine  Assembly,  70 1C 

2840-01-284-4011 

6071T24G01 

ENGINE,AIRCRAFT,TURBO-SHAFT 

Engine  Output  Drive  Shaft 

2835-01-123-7648 

70361-08004-043 

DRIVE  SHAFT  AS  SEMBL Y.ROT ARY  WING 

Intermediate  Gear  Box 

1615-01-074-5152 

70357-06300-042 

GEAR  BOX  ASSEMBLY 
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Depicted  below  in  Figure  2  is  the  location  of  these  components  on  the  UH-60  extracted 
from  Figure  2-2  of  TM  1-1520-237-10.  The  number  in  parenthesis  indicates  the  total  number  of 
that  type  of  component  on  the  aircraft. 

Figure  2.  Location  of  UH-60  Blackhawk  CBM+  Components 


3.  CH-47  Chinook  Helicopter  Information 

The  CH-47  Chinook  is  a  twin-turbine  engine,  tandem-rotor  helicopter.  Its  primary 
mission  is  the  transportation  of  cargo,  troops,  and  weapons  during  day,  night,  visual,  and 
instrument  conditions.  (TM  1-1520-240-10,  2003).  There  are  currently  sixteen  different 
components  of  four  different  types  on  the  CH-47  monitored  by  CBM+.  A  listing  of  CH-47 
Chinook  CBM+  components  is  listed  in  Table  3. 

Table  3.  CH-47  Chinook  CBM+  Component  Listing 


CH-47  COMPONENT 

NOMENCLATURE 

NSN 

PART  NUMBER 

FEDLOG  NOMENCLATURE 

Hinge  Pin  Assembly 

5315-01-295-7008 

114R2 197-7 

PIN,HOLLOW 

Tie  Bar  Assembly 

1615-00-740-6480 

1 14R2155-1 

TIE  BAR  ASSEMBLY,LAMINATED 

Engine,  Gas  Turbine 

2840-01-458-5361 

2-001-020-39 

ENGINE,AIRCRAFT,TURBO-SHAFT 

Fwd/Aft  Swashplate  Bearings 

3110-00-141-3750 

3110-01-356-0489 

1 14RS308-1 

1 14RS308-2 

SWASHPLATE  BEARING 
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Depicted  below  in  Figure  3  is  the  location  of  these  components  on  the  CH-47  extracted 
from  Figure  2-1-1  of  TM  1-1520-224-10.  The  number  in  parenthesis  indicates  the  total  number 
of  that  type  of  component  on  the  aircraft. 

Figure  3.  Location  of  CH-47  Chinook  CBM+  Components 


B.  SPECIFIC  DESCRIPTION  OF  COMPONENT  CHARACTERISTICS 

Brief  descriptions  of  the  components  of  interest  on  the  AH-64  Apache  referenced  from 
TM  1-1520-251-10  are: 

1.  Auxiliary  Power  Unit  Clutch  is  a  subcomponent  of  the  Auxiliary  Power  Unit  (APU) 
which  provides  both  hydraulic  pressure,  pressurized  air,  and  electrical  power  for  the  operation  of 
systems  onboard  the  AH-64  whether  the  engines  are  operating  or  not.  The  APU  is  required  to 
start  the  main  engines  unless  the  AH-64  is  assisted  with  an  Auxiliary  Ground  Power  Unit 
(AGPU). 

2.  70 1C  Engine  Assemblies  are  the  main  engines  for  the  AH-64.  The  engines  are  front 
drive  turbo  shaft  engines  of  modular  construction.  One  horizontally  mounted  engine  is  housed 
on  either  side  of  the  AH-64  aft  of  the  main  transmission  above  the  wing. 

3.  Utility  Hydraulic  Pump  is  a  subcomponent  of  the  utility  hydraulic  system  that 
provides  hydraulic  power  to  the  flight  controls,  weapon  drives,  ammunition  systems,  and 
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emergency  hydraulic  systems.  This  pump  is  mounted  on  the  accessory  drive  case  of  the  main 
transmission  (right  side). 

4.  Forward  Hanger  Bearing  is  a  component  of  the  tail  rotor  drive  system.  A 
hanger  bearing  supports  the  two  longest  shafts  of  three  shafts  that  lead  from  the  transmission  to 
the  intermediate  gear  box.  There  is  a  fourth  shaft  that  leads  from  the  intermediate  gear  box  to  the 
tail  rotor.  The  forward  hanger  bearing  is  located  on  the  end  of  the  second  shaft. 

5.  Aft  Hanger  Bearing  serves  the  same  purpose  as  the  Forward  Hanger  Bearing;  it  is 
located  on  the  end  of  the  third  shaft. 

6.  Main  Rotor  Pitch  Housing  is  a  subcomponent  of  the  rotor  head.  The  pitch  housing 
permits  blade  pitch  changes  in  response  to  flight  control  movements  transmitted  through  the 
swashplate. 

7.  Main  Rotor  Upper  Mast  Bearing  is  a  subcomponent  of  the  AH-64  mast  collocated 
with  the  rotor  head. 

8.  Main  Rotor  Lower  Mast  Bearing  is  similar  to  the  upper  bearing  but  located  lower. 

Brief  descriptions  of  the  components  of  interest  on  the  UH-60  Blackhawk  referenced 
from  TM  1-1520-237-10  are: 

1.  Oil  Cooler  Fan  Bearing  is  a  subcomponent  of  the  tail  rotor  drive  section.  The  oil 
cooler  cools  oil  from  the  engine  before  it  returns  it  to  the  oil  tank.  Shafts  from  the  main 
transmission  connect  the  oil  cooler  and  also  transmit  torque  to  the  tail  rotor.  There  are  four 
points  were  viscous  damped  bearings  are  mounted  on  adjustable  plates  which  support  these 
shafts  that  lead  to  the  tail  rotor. 

2.  Main  Rotor  Blades  are  a  subsystem  of  the  main  rotor  system.  A  rotor  blade  has  a 
titanium-spar  and  is  attached  to  spindles  which  are  retained  by  elastomeric  bearings  contained  in 
one-piece  titanium  hub.  The  elastomeric  bearing  permits  the  blade  to  flap,  lead,  and  lag. 

3.  Pump  Module  Assembly  is  a  component  that  provides  hydraulic  pressure  to  the 
Blackhawk’ s  hydraulic  system.  The  hydraulic  pump  module  assemblies  are  a  combination  of  a 
hydraulic  pump  and  a  hydraulic  fluid  reservoir. 
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4.  Damper  Assembly  is  located  between  the  main  rotor  blade  and  the  main  rotor  head. 
Main  rotor  dampers  are  installed  between  each  of  the  main  rotor  spindles  modules  and  the  hub  to 
restrain  leading  and  lagging  motions  of  the  main  rotor  blades  during  rotation  and  to  absorb  rotor 
head  loads  when  starting  the  aircraft.  Each  damper  has  a  small  hydraulic  fluid  reservoir. 

5.  70 1C  Engine  Assemblies  are  the  main  engines  for  the  UH-60.  The  engines  are  front 
drive  turbo  shaft  engines  of  modular  construction.  One  horizontally  mounted  engine  is  housed 
on  either  side  of  the  UH-60.  These  are  the  same  engines  that  are  mounted  on  the  AH-64  Apache. 

6.  Engine  Output  Drive  Shaft  is  a  subcomponent  of  the  UH-60  power  train  system.  It 
transfers  torque  generated  by  the  engine  to  the  main  transmission. 

7.  Intermediate  Gear  Box  is  a  subcomponent  of  the  UH-60  power  train  system.  It  is 
mounted  at  the  base  of  the  tail  pylon.  It  transmits  torque  and  reduces  shaft  speed  from  the  main 
module  gear  box  to  the  tail  rotor  gear  box. 

Brief  descriptions  of  the  components  of  interest  on  the  CH-47  Chinook  referenced  from 
TM  1-1520-224-10  are: 

1 .  Hinge  Pin  Assembly  is  a  component  of  the  rotor  system.  The  rotor  head  consists  of  a 
hub  connected  to  three  pitch-varying  shafts  by  three  horizontal  hinge  pins.  These  pins  permit 
blade  flapping.  Stops  on  the  top  and  bottom  of  the  hub  limit  the  blade  flapping  motion. 

2.  Tie  Bar  Assembly  is  located  close  to  the  hinge  pin  assembly.  It  connects  the  pitch- 
varying  shafts  to  the  pitch-varying  housings  on  the  rotor  heads. 

3.  Engines  on  the  CH-47  are  housed  in  separate  nacelles  mounted  externally  on  each  side 
of  the  aft  pylon. 

4.  Forward/Aft  Swashplate  Bearings  rotate  and  transfer  blade  pitch  changes  by  the  three 
pitch-varying  links  to  the  pitch-varying  housing  on  each  rotor  blade. 

C.  MAINTENANCE  REGIMES  OF  COMPONENTS 

Army  Aviation  maintenance  regimes  are  composed  of  both  periodic  and  on  condition 
maintenance  tasks.  Prescribed  maintenance  tasks  can  be  subdivided  into  five  major  areas: 
(AMCOM  Proof  of  Principle,  2005  ) 
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1.  PM:  Maintenance  or  inspections  performed  in  accordance  with  normal  Preventive 
Maintenance  Checks  and  Services. 

2.  On  Condition:  Maintenance  or  inspections  occurring  after  the  aircraft  encounters  a 
specific  event  or  flight  in  certain  environmental  conditions. 

3.  -18:  Maintenance  or  inspections  prescribed  in  maintenance  manuals  and  tracked  on  the 
- 1 8  forms  kept  in  the  aircraft  log  book. 

4.  ASAM:  Maintenance  or  inspections  listed  in  a  specific  Aviation  Safety  Action 
Message. 

5.  AWR:  Maintenance  or  inspections  listed  in  a  specific  Air  Worthiness  Release. 

These  maintenance  tasks  are  outlined  by  several  different  maintenance  manuals,  log  book 
forms,  and  messages/releases  specific  for  each  airframe  type.  A  component  may  be  inspected  as 
often  as  every  day  or  at  intervals  of  several  hundred  hours.  Each  inspection  interval  is  a  unique 
inspection  in  the  sense  that  some  of  the  more  common  inspections  are  a  visual  exterior  check  of 
a  component  whereas  the  more  infrequent  inspections  require  removal  of  the  component  from 
the  airframe,  disassembling  it,  and  conducting  a  much  more  thorough  inspection.  Each 
inspection  has  a  specific  number  of  maintenance-man  hours  (MMH)  required  in  order  to 
complete  the  maintenance  task. 

Listed  below  in  Tables  4  and  5  are  the  most  time  consuming  and  thorough  inspections  for 
each  component  listed  above  for  the  AH-64  Apache  and  UH-60  Blackhawk.  In  cases  where 
there  are  two  different  interval  inspections  with  the  same  MMH  and  clock  time  requirements  the 
more  frequent  inspection  of  the  two  is  listed: 
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Table  4.  AH-64  Apache  Maintenance  Regime 


AH-64  Component 

Inspection  Interval 

Reference 

Maint  Type 

MMH 

APU  Clutch 

250  Flight  Hours 

TB  1-1520-238-20-139 

-18 

34.4* 

Eng  Assy,  70 1C 

500  Flight  Hours 

TM  1- 1520-23  8-PM 

PM 

20.5* 

Utility  Hydraulic  Pump 

250  Flight  Hours 

TB  1-1520-238-20-139 

-18 

1.1 

Forward  Hanger  Bearing 

500  Flight  Hours 

TM  1-1520-238-PM 

PM 

4.4* 

Aft  Hanger  Bearing 

500  Flight  Hours 

TM  1-1520-238-PM 

PM 

4.4* 

Main  Rotor  Pitch  Housing 

125  Flight  Hours 

TM  1-1520-238-23 

-18 

0.5* 

Main  Rotor  Upper  Mast  Bearing 

500  Flight  Hours 

TM  1-1520-238-PM 

PM 

20.4* 

Main  Rotor  Lower  Mast  Bearing 

500  Flight  Hours 

TM  1-1520-238-PM 

PM 

20.4* 

*  If  no  Maintenance  Allocation  Chart  (MAC)  data  was  available  or  seemed  suspect  a  review  of  the  appropriate 

techniques  were  conducted  and  a  time  was  allocated  by  AMCOM  Subject  Matter  Experts 

Table  5.  UH-60  Blackhawk  Maintenance  Regime 


UH-60  Component 

Inspection 

Reference 

Maint  Type 

MMH 

Oil  Cooler  Fan  Bearing 

700  Flight  Hours 

TM  1-1520-237-PMI 

PM 

11.3* 

Main  Rotor  Blade 

700  Flight  Hours 

TM  1-1520-237-PMI 

PM 

17.2 

Pump  Module  Assembly 

700  Flight  Hours 

TM  1-1520-237-PMI 

PM 

0.03 

Damper  Assembly 

700  Flight  Hours 

TM  1-1520-237-PMI 

PM 

2.5* 

Engine  Assembly,  70 1C 

700  Flight  Hours 

TM  1-1520-237-PMI 

PM 

12.4* 

Engine  Output  Drive  Shaft 

700  Flight  Hours 

TM  1-1520-237-PMI 

PM 

4.9* 

Intermediate  Gear  Box 

120  Flight  Hours 

TM  1-1520-237-23 

-18 

2.0* 

*  If  no  Maintenance  Allocation  Chart  (MAC)  data  was  available  or  seemed  suspect  a  review  of  the  appropriate  techniques 

were  conducted  and  a  time  was  allocated  by  AMCOM  Subject  Matter  Experts 

A  CH-47  Chinook  Maintenance  Regime  is  not  listed  since  a  Proof  of  Principle  brief  was 
not  conducted  on  the  CH-47  Chinook  CBM+  program  due  to  the  progression  of  the  CH-47 
CBM+  program.  Proofs  of  Principle  briefs  are  the  source  of  maintenance  regime  information  for 
both  the  AH-64  Apache  and  UH-60  Blackhawk.  (Brown,  2005) 

D.  FAILURE/AGE  REPLACEMENT  CHARACTERISTICS  OF  COMPONENTS 

The  Aviation  and  Missile  Research,  Development,  and  Engineering  Center  (AMRDEC) 
provided  data  sets  for  component  failure  times.  The  failure  times  are  the  occurrences  of  serious 
diagnostic  symptoms  requiring  maintenance  actions.  The  data  sets  are  displayed  in  Microsoft 
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Excel  format  and  each  data  set  consists  of  twenty-one  columns.  The  twenty-two  columns  are 
listed  below. 

1.  WUC:  Work  Unit  Code 

2.  PN:  Part  Number  of  component 

3.  SN:  Serial  Number  of  component 

4.  El  SN:  End  Item  Serial  Number  (aircraft  tail  number) 

5.  MODEL:  The  model  of  aircraft 

6.  DATE:  Date  the  action  was  performed 

7.  REP  NUM:  Repair  Number-The  number  of  times  the  part  was  removed  for  a  causable 
removal. 

8.  CEN:  Censored-If  1  then  installed  and  still  flying,  else  0  and  component  removed 

9.  LIFE:  If  CEN=0,  the  time  on  the  component  since  new  or  last  causable  removal.  If 
CEN=1,  the  current  time  on  the  component. 

10.  TSN:  Time  Since  New 

1 1 .  TSO:  Time  Since  Overhaul 

12.  NOVH:  Number  of  Overhauls 

13.  F_TYPE:  Type  of  failure 

14.  FCODE:  Failure  Code  001  to  999,  the  reason  the  part  was  removed 

15.  FAILURE:  Narrative  for  the  FCODE 

16.  FAMILY:  The  fail  code  grouped  into  failure  family  types. 

17.  PREV  FC:  Failure  Code  on  the  previous  removal 

18.  REP_UIC:  UIC  that  repaired  the  item  last  or  original  manufacture  if  REP_NUM=1. 

19.  REPAIR:  Location  name  of  the  UIC  that  repaired  the  item  last. 

20.  UIC:  Unit  Identification  Code  of  removing  unit 

2 1 .  LOC :  Location  of  the  UIC 
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22.  SNPREFX:  Serial  Number  substring  as  a  prefix  to  perform  comparison  analysis 

In  addition  to  this  data,  each  airframe  type  has  a  Technical  Manual  which  states  a 
component’s  Time  Between  Overhaul  (TBO)  and/or  component  retirement  time.  These  manuals 
allow  the  determination  of  a  component’s  age  replacement  time  if  one  exists.  Listed  below  in 
Table  6  are  the  CBM+  component’s  estimated  mean  time  between  failure  (MTBF)  and  Age 
Replacement  times.  The  Age  Replacement  times  are  derived  from  the  applicable  airframes’ 
Technical  Manual.  The  MTBFs  listed  below  are  obtained  from  AMCOM’s  Proof  of  Principle 
briefs  prepared  in  July  2005.  (Brown,  2005)  The  MTBF  estimates  take  into  account  that  some  of 
the  data  are  censored.  When  both  TBO  and  component  retirement  times  are  given  by  the 
applicable  reference  the  more  restrictive  of  the  two  numbers  is  listed. 

Table  6.  CBM+  Component  MTBF  and  Age  Replacement 


AH-64  Component 

MTBF 

Age  Replacement 

APU  Clutch 

900  Flight  Hours 

On  Condition  of  Failure 

Engine  Assembly,  70 1C 

1,304  Flight  Hours 

5,000  Flight  Hours** 

Utility  Hydraulic  Pump 

464  Flight  Hours 

On  Condition  of  Failure 

Forward  Hanger  Bearing 

834  Flight  Hours 

2,500  Flight  Hours 

Aft  Hanger  Bearing 

537  Flight  Hours 

2,500  Flight  Hours 

Main  Rotor  Pitch  Housing 

257  Flight  Hours 

5,300  Flight  Hours 

Main  Rotor  Upper  Mast  Bearing 

1.250  Flight  Hours 

1,750  Flight  Hours 

Main  Rotor  Lower  Mast  Bearing 

1.250  Flight  Hours 

9,400  Flight  Hours 

UH-60  Component 

MTBF 

Age  Replacement 

Oil  Cooler  Fan  Bearing 

14,425  Flight  Hours 

700/2,100  Flight  Hours* 

Main  Rotor  Blade 

1.594  Flight  Hours 

9,600  Flight  Hours 

Pump  Module  Assembly 

5,969  Flight  Hours 

On  Condition  of  Failure 

Damper  Assembly 

6,629  Flight  Hours 

On  Condition  of  Failure 

Engine  Assembly,  70 1C 

1,342  Flight  Hours 

5,000  Flight  Hours** 

Engine  Output  Drive  Shaft 

2,581  Flight  Hours 

On  Condition  of  Failure 

Intermediate  Gear  Box 

6,624  Flight  Hours 

On  Condition  of  Failure 

CH-47  Component 

MTBF 

Age  Replacement 

Hinge  Pin  Assembly 

3,393  Flight  Hours 

1,200  Flight  Hours 

Tie  Bar  Assembly 

2,925  Flight  Hours 

4,800  Flight  Hours 

Engine 

602  Flight  Hours 

2,400  Flight  Hours 

Forward/Aft  Swashplate  Bearings 

991/867  Flight  Hours 

1,200  Flight  Hours 

*700  flight  hours  when  installed  at  Station  410.5;  all  others  2,100  flight  hours 

**701C  Engine  Assembly  has  several  components  changed  at  various  intervals.  5,000 

Flight  Hours  is  the  most  common  age  replacement  time  of  the  most  critical  components. 
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E.  COSTS  ASSOCIATED  WITH  COMPONENTS 


Listed  below  are  each  component’s  cost  and  an  estimate  of  its  shipping  cost.  It  is 
assumed  that  the  components  can  be  divided  into  three  different  weight  classifications.  A 
component  weight  is  considered  to  be  either  light,  medium,  or  heavy.  In  addition,  the 
component’s  shipment  is  either  urgent  or  not  urgent.  Table  7  displays  the  approximate  costs  of 
shipping  components  of  different  weights  2000  miles  under  urgent  and  not  urgent  requirements. 
These  shipping  costs,  while  approximate,  are  realistic.  Although  the  model  and  simulation  do 
not  include  costs  for  component  shipping  this  must  occur  in  both  the  Legacy  and  CBM+  Process; 
it  is  important  to  state  these  values  in  recognition  that  this  exists  in  both  processes. 

Table  7.  CBM+  Component  and  Shipping  Costs 


AH-64  Component 

FEDLOG  Unit 

Price 

OSMIS  Weight 

Weight 

Classification 

Urgent  Shipping 

Cost 

Non-urgent 

Shipping  Cost 

APU  Clutch 

$27,774.00 

Unavailable 

Medium 

$1,000 

$500 

Engine  Assembly,  70 1C 

$720,974.00 

Unavailable 

Heavy 

$3,000 

$1,500 

Utility  Hydraulic  Pump 

$9,437.00 

Unavailable 

Medium 

$1,000 

$500 

Forward  Hanger  Bearing 

$6,941.00 

400 

Light 

$200 

$50 

Aft  Hanger  Bearing 

$5,673.00 

216 

Light 

$200 

$50 

Main  Rotor  Pitch  Housing 

$6,846.00 

894 

Medium 

$1,000 

$500 

Main  Rotor  Upper  Mast  Bearing 

$7,295.00 

275 

Light 

$200 

$50 

Main  Rotor  Lower  Mast  Bearing 

$5,126.84 

40 

Light 

$200 

$50 

UH-60  Component 

FEDLOG  Unit 

Price 

OSMIS  Weight 

Weight 

Classification 

Urgent  Shipping 

Cost 

Non-urgent 

Shipping  Cost 

Oil  Cooler  Fan  Bearing 

$290.71 

8 

Light 

$200 

$50 

Main  Rotor  Blade 

$130,420.00 

6500 

Medium 

$500 

Pump  Module  Assembly 

$16,771.00 

560 

Medium 

$500 

Damper  Assembly 

$9,770.00 

500 

$200 

$50 

Engine  Assembly,  70 1C 

$720,974.00 

Unavailable 

Heavy 

$3,000 

$1,500 

Engine  Output  Drive  Shaft 

$4,812.00 

Unavailable 

Medium 

$1,000 

$500 

Intermediate  Gear  Box 

$20,694.00 

800 

Medium 

$1,000 

$500 

CH-47  Component 

FEDLOG  Unit 

Price 

OSMIS  Weight 

Weight 

Classification 

Urgent  Shipping 

(  ost 

Non-urgent 

Shipping  Cost 

Hinge  Pin  Assembly 

330 

Light 

$200 

$50 

Tie  Bar  Assembly 

120 

Light 

$200 

$50 

Engine 

11,650 

Heavy 

$3,000 

$1,500 

Forward/Aft  Swashplate  Bearings 

Unavailable 

Unavailable 

Light 

$200 

$50 
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III.  STOCHASTIC  MODELS  FOR  COMPARING  LEGACY 
MAINTENANCE  AND  CONDITION-BASED  MAINTENANCE  PLUS 

A.  IMPORTANT  FACTORS  FOR  CBM+  CANDIDACY 

In  determining  a  whether  or  not  to  introduce  a  component  into  the  CBM+  program  a 
number  of  factors  should  be  considered: 

1 .  How  often  does  the  component  fail  during  active  flight  hours? 

It  is  preferable  to  introduce  a  component  that  fails  often  versus  components  that  rarely 

fail. 

2.  What  are  the  consequences  of  a  component  failing? 

It  is  preferable  to  introduce  a  component  that  possesses  severe  consequences  for  failure 
versus  a  component  that  is  inconsequential  in  its  failure. 

3.  What  is  the  difficulty  level  of  the  inspection  that  would  be  alleviated  by  CBM+? 

If  an  inspection  is  extremely  difficult  and  resource  intensive  to  perform  it  would  be 
preferable  to  have  this  inspection  replaced  by  CBM+  versus  an  inspection  that  is  simple  and  easy 
to  perform. 

4.  Is  a  CBM+  sensor  feasible  for  a  component? 

If  measurements  cannot  be  taken  to  assess  the  degradation  of  the  component  before  it 
fails  then  that  component  would  not  be  a  good  CBM+  candidate.  Furthermore,  the  time  from 
occurrence  of  measurable  evidence  of  impending  failure  until  failure  should  be  long  enough  to 
detect  the  impending  failure  and  take  maintenance  action. 

5.  What  is  the  reliability  of  the  CBM+  sensor,  the  difficulty  of  inspecting  the  CBM+ 
sensor,  and  the  effect  of  the  CBM+  sensor  on  performance  of  the  component  being  monitored? 
Is  there  even  a  CBM+  sensor  currently  developed  for  a  component?  What  is  the  cost  of  the 
sensor? 

By  focusing  on  the  CBM+  sensor  it  can  be  determined  whether  or  not  the  sensor 
itself  may  sometimes  miss  impending  failures,  give  false  positives,  require  intensive 
maintenance,  or  impede  standard  operations.  Maintainers  are  often  wary  of  adding  a  new  system 
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designed  to  help  them  conduct  maintenance  operations  for  fear  of  there  now  being  one  more 
system  to  maintain. 

Stochastic  models  can  be  created  to  assist  in  identifying  information  needed  to  determine 
characteristics  of  components  that  make  the  component  a  good  candidate  for  monitoring  by 
CBM+.  Such  stochastic  models  compare  a  single  component’s  performance  and  associated  costs 
in  both  the  CBM+  and  Legacy  processes.  The  models  can  vary  in  detail  and  complexity; 
transparency  and  simplicity  are  desirable. 

B.  MODEL  OVERVIEW 

In  choosing  a  model  to  represent  the  operating  environment  of  CBM+  it  is  convenient  to 
let  every  aircraft  component  being  considered  for  CBM+  begin  in  good  operating  condition. 
Over  time  the  component’s  condition/state  degrades  and  eventually  the  component  fails.  We 
consider  a  component  that  allows  predictive  measurements  to  be  made  of  its  condition  (failure 
propensity);  these  are  diagnostic  symptoms  (DS).  If  a  DS  can  be  recognized  by  maintenance 
personnel  then  this  information  is  a  useful  indication  that  the  component  is  beginning  to  fail.  A 
DS  can  be  detected  by  a  CBM+  sensor  or  noticed  by  a  maintainer  during  the  conduct  of 
scheduled  maintenance,  or  by  the  operator. 

If  conditions  for  a  near-tenn  future  failure  exist  it  is  essential  to  recognize  the 
impending  failure  in  order  to  lessen  such  a  failure’s  effect.  CBM+  sensors  recognize  impending 
failure  of  components  by  monitoring  the  component’s  performance.  Specifically  the  UH-60 
IMD-HUMS  system  notes  condition  indicators  (CIs)  based  on  vibratory  analysis  of  items  such  as 
bearings,  shafts,  and  gears.  These  CIs  are  rolled  up  into  health  indicators  (His)  which  are 
numbers  from  0  to  1  displaying  the  perceived  health  of  that  component.  (Wright,  2005)  This 
system  of  CIs  and  His  is  used  to  determine  the  occurrence  of  DSs.  Under  the  Legacy 
maintenance  system  when  maintenance  personnel  conduct  a  scheduled  inspection  of  an  aircraft 
they  are  looking  for  chips,  cracks,  dents,  nicks,  wears,  incorrect  lubrication  levels,  incorrect 
pressure  outputs  (pneumatic  and  fluids),  and  other  potential  faults.  These  physical  inspections 
determine  the  occurrence  of  DSs.  A  DS  may  or  may  not  be  noticed  during  the  downtime 
immediately  following  the  mission  during  which  it  was  generated.  All  DSs  are  discovered  during 
inspections  during  downtimes.  (Gaver  and  Jacobs,  2006) 
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The  following  are  the  components  which  form  a  general  framework  for  a  Non- 
Homogeneous  Poisson  Process  (NHPP)  model  for  the  occurrence  of  DSs  for  both  the  CBM+ 
process  and  the  Legacy  process: 

N(t)  :  number  of  transitions  of  components  from  good  condition  to  poor  condition 
during  the  time  interval  (0,/],  (number  of  occurrences  of  DSs  during  (0,t]);  each  transition 
corresponds  to  the  occurrence  of  one  DS.  In  many  cases  the  times  between  failures  of  repairable 
components  may  tend  to  decrease  as  the  components  age.  Thus  j N (/);/>  Oj  is  assumed  to  be  a 
nonhomogeneous  Poisson  process  with 

A(/) :  mean  value  function  of  {N(t);t>0};  that  is  E^N (/)]  =  A(/) . 


X{t) :  intensity  function  of  {N(t);t>0};  that  is  X{t ) 


dh(t) 

dt 


m:  constant  mission  length 

C.  A  NON-HOOMOGENEOUS  POISSON  PROCESS  (NHPP)  MODEL 

Renewal  processes  are  often  used  to  model  times  between  failures  for  a  system.  For  a 
renewal  process  to  apply,  the  times  between  successive  failures  should  be  independent  and 
identically  distributed  with  an  arbitrary  distribution.  (Ross,  2003)  In  order  to  either  accept  or 
refute  the  assumption  that  the  times  between  failures  are  independent  and  identically  distributed 
failure  data  must  be  analyzed.  The  data  set  that  was  selected  for  evaluation  is  the  set  of  AH- 
64/UH-60  70 1C  Engine  lifetime  data.  The  lifetimes  are  the  times  between  occurrences  of 
serious  diagnostic  symptoms  requiring  maintenance  action.  The  reason  for  the  selection  of  this 
data  set  is  that  each  engine  may  have  several  failure  times  recorded.  The  data  set  contains  both 
censored  and  uncensored  lifetime  data.  Multiple  lifetimes  for  an  engine  are  indicated  by  the 
same  engine  serial  number  (SN)  being  listed  with  consecutive  lifetimes  (REP_NUM).  This  gives 
the  opportunity  to  assess  whether  or  not  the  successive  times  between  failures  for  an  engine  can 
be  approximately  represented  as  independent  and  identically  distributed.  If  there  is  evidence  that 
the  successive  lifetimes  are  not  independent  and  identically  distributed,  then  a  renewal  process 
model  for  the  times  between  failures  is  not  appropriate. 
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Analysis  of  the  engine  data  suggests  that  the  times  between  failures  are  not  identically 
distributed.  Therefore  a  Non-Homogeneous  Poisson  Process  (NHPP)  model  for  the  failure  times 
is  considered.  Although  the  term  “failure”  is  used,  these  events  are  actually  occurrences  of 
serious  diagnostic  symptoms  that  require  maintenance  action. 


The  AH-64/UH-60  701C  Engine  data  provided  by  AMRDEC  consists  of  3,385  entries  in 
the  format  outlined  in  Chapter  Two.  From  this  group  of  3,385  entries  there  were  2,045  entries 
that  were  first  lifetimes,  913  entries  that  were  second  lifetimes,  480  that  were  third  lifetimes,  and 
397  that  were  fourth  lifetimes.  These  data  are  not  complete  with  regards  to  all  lifetimes  being 
annotated  for  all  serial  numbers.  There  are  missing  lifetime  data  which  are  not  due  to  censoring. 
However,  some  of  these  missing  lifetimes  can  be  inferred  from  other  recorded  data.  For 
example,  suppose  the  first  lifetime  is  missing  but  the  second  lifetime  is  recorded  along  with  the 
time  since  the  engine  was  new  at  the  time  of  the  second  failure.  In  this  case  the  first  lifetime  can 
be  inferred  by  subtracting  the  second  lifetime  from  the  time  since  new.  The  engine  data 
considered  appear  in  Appendix  A.  The  engines  considered  have  at  least  3  failure  times  recorded. 
The  columns  named  WUC,  PN,  TSO,  NOVH,  REP  UIC,  REPAIR,  UIC,  LOC,  and  SN_PREFX 
have  been  omitted  from  the  original  data  set  since  these  factors  are  not  relevant  in  this  analysis, 
and  to  make  the  data  set  more  compact.  None  of  the  engines  displayed  TSO  (Time  Since 
Overhaul)  and  NOVH  (Number  of  Overhauls);  therefore  these  columns  are  omitted. 

In  Figure  4  the  mean  times  between  failures  for  the  first,  second,  third  and  fourth  failures  with 
95%  confidence  intervals  are  displayed;  censored  lifetimes  are  not  included.  This  figure 
suggests  that  the  successive  lifetimes  of  an  engine  are  not  identically  distributed;  the  time  until 
first  failure  tends  to  be  much  larger  than  the  subsequent  times  between  failures. 

Figure  4.  Mean  Failure  Times  for  Engines  by  Consecutive  Lifetimes 


Mean  Failure  Times  for  Engines  by  Consecutive 
Lifetimes 


Fifth  Lifetime 
Fourth  Lifetime 
Third  Lifetime 
Second  Lifetime 
First  Lifetime 


0  500  1000  1500  2000 

Engine  Lifetimes  (Operating  Hours) 


52 


An  estimate  of  the  intensity  fhnction  of  a  NHPP  is  obtained  by  using  disjoint  time 
intervals  of  length  500  hours  from  0  to  3,000  hours.  Let  AL  (/)  be  the  number  of  failures  for 

engine  j  during  the  time  interval  (0,t], 

f  1  if  the  last  observation  for  engine  j  (censored  or  not)  is  greater  than  x 

I  Ax,  oo)  =  < 

[0  otherwise 

The  intensity  function  in  the  age  interval  [x,  x+500]  where  x  e  (0,  500,  1,000,  1,500,  etc.)  is 
estimated  as 

X[V;(500  +  Jr)-JV,.(*)] 

— -  (Jacobs,  2006) 

i00Yl,(x) 

j 

A  display  of  the  resulting  estimated  intensity  function  is  in  Appendix  B.  The  log  of  the 
estimated  intensity  function  versus  log  t  is  also  displayed.  This  latter  display  suggests  that  an 

NHPP  with  power-law  mean  value  function  A(t)  =  ytd  tends  to  summarize  the  data  well.  The 

parameters  of  a  NHPP  with  power-law  mean  value  function  of  the  form  A  (t)  =  yr  are 

estimated  from  the  engine  data  using  maximum  likelihood.  The  estimates  and  standard  errors 
are  listed  in  Figure  5.  The  standard  errors  are  obtained  using  Fisher  information.  (Bickel  and 
Doksum,  1977) 


Figure  5.  NHPP  Parameter  Estimates 


NHPP  Parameter  Estimates 

Estimate  of  y 

5.4x1  O'6 

Estimate  of  5 

1.71 

Standard  Error 

6.3x10’7 

Standard  Error 

0.015 

Estimate  ±  2  s.e. 

mi  m  hytawi 

Estimate  ±  2  s.e. 

(1.68,1.74) 

The  estimate  of  <5  is  statistically  significantly  greater  than  1,  suggesting  that  failures  are  more 
frequent  for  older  engines.  The  older  an  engine,  the  more  likely  various  subcomponents  are  to 
fail. 

D.  MODEL  STRUCTURE 

The  power  law  NHPP  model  with  the  estimated  parameters  is  used  in  a  model  to  assist  in 
the  selection  of  components  for  inclusion  in  the  CBM+  process.  The  model  represents  the  arrival 
of  DSs  during  each  mission,  the  number  of  downtimes  elapsed  until  the  DSs  are  discovered,  the 
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repair  times  associated  with  the  discovery  of  the  DSs,  and  the  chance  a  component  failure  will 
cause  a  mission  abort.  This  analytical  model  is  taken  directly  from  Gaver  and  Jacobs  (2006). 

Let  TV(/)be  the  number  of  transitions  of  components  from  good  condition  to  poor 
condition  (occurrence  of  diagnostic  symptoms  (DS))  during  the  operational  time  interval  (0,t], 
{TV (/);/  >  0|  is  a  nonhomogeneous  Poisson  process  with  mean  value  function  A(/)  =  £’[Ar  (/)] 


and  intensity  function  A  (/)  =  — - —  .  The  DSs  may  not  be  discovered  immediately.  The  DSs  are 

discovered  during  inspection  during  downtimes.  Each  mission  is  of  length  m.  Let 
Nj  =  N  (mi)-  N  { /»(/'  - 1 )  )  be  the  number  of  DSs  to  occur  during  the  ilh  mission.  TV,  has  a 

Poisson  distribution  with  mean  A,  =  A(/m/)-A(/m(/-1))  .  For  example  if  the  length  of  the 


mission  is  4  hours  long  (m= 4)  andA(/)  =  yis ,  then  A,- =  ^(4/)d -^(4(/-l))d  .  The 
computations  used  to  derive  the  estimates  of  y  and  5  are  listed  in  Appendix  C.  These  estimated 
values  are  used  throughout  the  thesis. 

Let  Pj  be  the  conditional  probability  a  DS  is  discovered  during  the  /th  downtime  after  the 

mission  within  which  it  first  appears,  given  it  has  not  been  discovered  before  and  the  DS  has  not 
caused  a  mission  abort;  the  downtime  immediately  after  the  mission  the  DS  appears  in  is  labeled 
downtime  1.  In  general,/^  < p2  <.. . .  Assume  that  whether  or  not  a  DS  is  discovered  during  a 

downtime  is  independent  from  downtime  to  downtime  and  from  DS  to  DS.  Suppose  also  that  if  a 
DS  is  generated  during  a  mission  it  can  critically  activate  during  that  mission,  causing  the 
mission  to  fail.  This  is  known  as  a  mission  abort.  Further,  let  denote  the  conditional 

probability  that  the  DS  critically  activates  during  the  /th  mission  after  its  generation,  given  it  has 
not  been  discovered  and  rectified  in  advance;  a{  is  the  probability  of  critical  activation  during 


the  mission  of  its  genesis.  Let  Dii+-  be  the  number  of  DSs  that  occurred  during  mission  /that 
are  discovered  during  the  /th  downtime  after  mission  /before  a  critical  activation;  it  has  a 


Poisson  distribution  with  mean  A, 


nu-zAK1-,) 

k=i 


( 1  -  ct  j  j  p j ;  that  is,  I), ,  has  a  Poisson 


distribution  with  mean  A,  (I  -  a, ) P\ ;  Dii+l  has  a  Poisson  distribution  with  mean 
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Az(l-a1)(l-/»1)(l-a2)/>2^  etc.  Further,  these  random  variables  are  currently  assumed 
independent.  It  is  of  course  possible  that  critical  activation  occurs  before  the  DS  is  discovered 
and  the  fault  removed.  Let  Dj  i+J  denote  the  number  of  DSs  that  are  generated  during  the  7th 
mission  that  give  rise  to  a  critical  activation  (mission  abort)  before  discovery  j  missions  after 
they  are  generated.  Clearly  D ) ;  is  Poisson  with  mean  Ay/, ,  A)  /+1  is  Poisson  with  mean 
A,  (l-fl1)(l-/71)a2  ;  etc.  (Gaver  and  Jacobs,  2006) 

Assume  that  DSs  that  cause  mission  abort  are  discovered  in  the  downtime  following  the 
aborted  mission.  The  number  of  DSs  discovered  during  the  ith  downtime, 

i  _  _ 

Dj  =  2_,  Dj  j+a-j)  +  D j  j+d-j)  >  has  a  Poisson  distribution  with  mean 

j= 1 

£[A-]  =  ZAJ  Y\(l-Pk)(l-ak)  [l  —  ( 1  —  ai+l-j  )  ( 1  -  Pi+i-j  )_ 

7=1  L k={  J 

where  an  empty  product  is  interpreted  as  equal  to  1.  (Gaver  and  Jacobs,  2006) 

The  expected  number  of  missions  that  are  not  aborted  during  the  first  1  scheduled 
missions  can  be  calculated  as  follows.  Let  S ( 1 )  be  the  number  of  DSs  that  can  cause  mission 

abort  during  mission  1  ;  there  is  at  most  one  mission  abort  per  mission.  Let  s(r r)  be  the 

probability  a  DS  causes  a  mission  abortion  during  the  /7th  mission  after  it  is  generated.  Assume 
Pi  =  p  and  cij  =  a  for  all  i . 

5(77)  =  [(l-a)(1-p)]"  1  a  (1) 

£[‘s,(/)]=Zv('-(/-i)) 

7=1 

Far[5(0]  =  ZV(/-(z'-1)) 

7=1 
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S(t)  has  a  Poisson  distribution.  The  probability  mission  1  is  aborted  is 
Pj.S(/)  >  o]  =  1-exp  j-£[.S(/)]j .  Let  ^(/)be  the  number  of  missions  that  have  been  aborted 
during  the  first  t  missions.  A  mission  is  aborted  if  at  least  one  DS  causes  a  mission  abort. 

£[^(/)]  =  Z[1_exP{_£[‘S(i)]}l  (2) 

i=[  L 

VAR  [  A  (/)]  =  X  [i  -  exp  {-A [.S’  (/)]}]  exp  [-E [.S (/)]} 

i= 1 

In  the  general  case 

5(/)  =  n[(1-ar)(1-A)^  (3) 

i=l 

where  an  empty  product  is  set  equal  to  1.  (Gaver  and  Jacobs,  2006) 
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IV.  LEGACY  MAINTENANCE  AND  CONDITION-BASED 
MAINTENANCE  PLUS  SIMULATION 

A.  SIMULATION  ARCHITECTURE  AND  CHARACTERISTICS 

This  simulation  is  based  on  the  nonhomogeneous  Poisson  process  model  as  described  in 
Chapter  III.  The  Visual  Basic  for  Applications  (VBA)  coding  used  as  a  macro  within  Microsoft 
Excel  2003  is  listed  in  Appendix  D.  The  Excel  workbook  consists  of  four  worksheets  named 
“RVs”,  “Legacy  Process”,  “CBM+  Process”,  and  “Data”. 

The  purpose  of  “RVs”  worksheet  is  to  receive  all  variable  inputs.  The  variable  inputs  for 
this  simulation  are: 

1.  #  Replications:  This  number  sets  the  number  of  iterations  for  the  simulation  to 
perform. 

2.  Piih):  The  probability  that  the  Legacy  Process  will  recognize  a  DS  during  a  downtime 
that  includes  an  inspection  lasting  h  hours. 

3.  Pc'.  The  probability  that  the  CBM+  Process  will  recognize  a  DS  during  a  downtime. 
The  number  of  downtimes  until  a  DS  is  discovered  has  a  geometric  distribution  with  mean  . 

For  each  DS  an  independent  Pc  is  drawn  from  a  Beta  distribution  with  mean  0.99  and  variance 
.0003771 1.  The  randomization  of  Pc  is  determined  by  a  Beta  distribution  and  both  a%  and  fa 
parameters  are  entered  on  the  worksheet.  Initially  the  Beta  distribution  is  generated  using  o.b= 25 
and  fa=.25.  The  equation  for  the  probability  density  function  of  the  beta  distribution  is: 

f(x',aB,fiB)  = - - - for0<x<l  and  0  otherwise 

B(ccb,Pb) 

where  B  is  the  normalizing  constant. 

4.  R0:  The  initial  repair  time  incurred  upon  a  DS  discovery. 

5.  Rp  The  subsequent  repair  time  incurred  if  DS  not  discovered  after  first  mission. 

6.  A0:  The  initial  repair  time  incurred  upon  a  mission  abort. 
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7.  Ai:  The  subsequent  repair  time  incurred  if  mission  abort  causing  DS  is  not  discovered 
after  first  mission. 

8.  Ma:  For  each  DS  an  independent  time  until  the  DS  results  in  a  mission  abort  is  drawn 
from  a  Weibull  distribution  with  shape  parameter  1.5  and  mean  10.  The  values  of  both  Weibull 
parameters  aw  and  ftw  parameters  are  entered  on  the  worksheet.  Initially  the  Weibull  distribution 
is  generated  using  aw=  1.5  and  pw=  1 1.08.  The  equation  for  the  probability  density  function  of  the 
Weibull  distribution  is: 


/(x;  a,,. ,  Ar )  =  ’ 

Pw  Pw 


for  x  >  0  and  0  otherwise  . 


9.  PoH:  This  is  the  probability  that  the  required  replacement  component  is  on  hand  when 
a  DS  is  discovered  or  causes  mission  abort.  For  each  discovered  DS,  or  a  DS  that  causes  a 
mission  abort,  an  independent  Bernoulli  random  number  is  generated  to  determine  if  the 
replacement  component  is  immediately  available  or  must  be  ordered  from  a  depot. 

10.  Tom  If  a  replacement  component  is  ordered,  a  time  Toh  until  the  replacement 
component  arrives  is  generated;  the  time  has  an  exponential  distribution. 

11.  A.,:  The  expected  number  of  DSs  to  occur  during  mission  i  having  length  4  hours; 
Xl  =  ;/[(4/)'5  —  (4(/  —  l)'1 )] ;  the  values  of  gamma  and  delta  are  the  maximum  likelihood  estimates 
obtained  from  analysis  of  the  engine  data. 

The  “RVs”  worksheet  generates  the  number  of  DSs  to  occur  during  each  mission;  the 
number  of  DSs  that  occur  during  the  ith  mission  is  generated  using  a  Poisson  distribution  where 
the  mean  is  determined  by  the  appropriate  A.;  .  These  numbers  of  DSs  are  used  for  both  the 
“Legacy  Process”  and  the  “CBM+  Process”.  This  provides  a  common  arrival  process  for  both 
the  Legacy  and  CBM+  processes. 

The  purpose  of  the  “Legacy  Process”  and  “CBM+  Process”  worksheets  is  to  provide  the 
cell  structure  and  formula  to  determine  the  following  information  for  the  two  different  processes: 

1.  Dij:  The  number  of  downtimes  to  occur  until  the  jth  DS  generated  during  the  ith 
mission  is  discovered. 
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2.  MAij:  The  number  of  downtimes  to  occur  until  the  jth  DS  generated  during  the  ith 
mission  results  in  a  mission  abort;  for  each  DS  the  same  time  until  mission  abort  is  used  in  the 
Legacy  and  CBM+  models. 

3.  Rij:  The  total  repair  time  from  the  jth  DS  resulting  from  the  ith  mission. 

4.  Sij :  The  total  time  awaiting  replacement  components  from  the  jth  DS  resulting  from  the 
ith  mission;  for  each  DS  the  time  is  drawn  from  an  exponential  distribution  with  a  common  mean 
for  both  the  Legacy  and  CBM+  models. 

The  purpose  of  the  “Data”  worksheet  is  to  display  the  results  of  each  simulation 
replication  and  then  compute  the  means,  standard  deviations,  and  95%  confidence  intervals  of 
the  results  for  the  Legacy  and  CBM+  processes.  The  results  that  are  tabulated  for  both  processes 
are: 

1.  XE)Ss<=M:  The  number  of  diagnostic  symptoms  that  are  detected  before  the  end  of 
the  total  number  of  missions  observed  (M). 

2.  £DSs>M:  j[ie  11Umber  of  diagnostic  symptoms  that  are  detected  after  the  end  of  the 
total  number  of  missions  observed  (M)  but  were  generated  during  the  M  missions. 

3.  £Aborts<=M:  The  number  of  mission  aborts  that  occur  before  the  end  of  the  total 
number  of  missions  observed  (M). 

4.  £Aborts>M:  The  number  of  mission  aborts  that  occur  after  the  end  of  the  total 
number  of  missions  observed  (M)  that  are  due  to  DSs  generated  during  the  M  missions. 

5.  yR<=M:  The  sum  of  repair  times  that  occur  before  the  end  of  the  total  number  of 
missions  observed  (M). 

6.  £R>M:  The  sum  of  repair  times  that  occur  after  the  end  of  the  total  number  of 
missions  observed  (M)  that  are  due  to  DS  generated  during  the  M  missions. 

7.  £R:  The  sum  of  repair  times  that  occur. 

8.  yS<=M:  The  sum  of  time  spent  awaiting  component  arrival  that  occurs  before  the  end 
of  the  total  number  of  missions  observed  (M). 

9.  £S>M:  j[ie  sum  0f  time  spent  awaiting  component  arrival  that  occurs  after  the  end  of 
the  total  number  of  missions  observed  (M)  that  are  due  to  DS  generated  during  the  M  missions. 


59 


10.  VS:  The  sum  of  time  spent  awaiting  component  arrival  that  occurs. 

B.  DETERMINATION  OF  Pc  and  PL 

The  number  of  downtimes  until  the  Legacy  or  CBM+  process  recognizes  a  DS  is  a  direct 
result  of  a  process’s  probability  of  successfully  at  detecting  a  DS.  The  conditional  probability  a 
DS  is  discovered  n  downtimes  after  it  is  generated  in  the  Legacy  process,  given  it  has  not  been 
discovered  before  and  has  not  caused  a  mission  abort,  is  Pi{h)  where  h  is  the  maintenance  man¬ 
hours  (MMH)  incurred  during  the  downtime;  in  the  CBM+  process  the  conditional  probability  a 
DS  is  discovered  during  a  downtime,  given  it  has  not  been  discovered  before  and  has  not  caused 
a  mission  abort  is  a  constant  Pc',  that  is  the  number  of  downtimes  until  a  generated  DS  is 
discovered  has  a  geometric  distribution  with  probability  of  success  Pc-  Initially  it  is  assumed 
that  the  CBM+  Process  Pc  has  an  expected  value  of  0.99;  this  may  be  an  optimistic  value.  The 
effect  of  the  variability  of  the  time  to  discover  different  DSs  is  modeled  by  randomizing  Pc  using 
a  Beta  distribution.  For  each  generated  DS  the  Pc  is  independently  drawn  from  a  Beta 
distribution  having  mean  0.99.  The  mean  value  of  0.99  is  based  on  a  telephone  conversation 
with  Mr.  Johnny  Wright  the  Deputy  Program  Manger  for  B.F.  Goodrich  Corporation’s  IMD- 
HUMS  program  on  January  27,  2006.  (Wright,  Personal  Communication,  2006)  He  stated  that 
the  parameters  of  the  CBM+  sensors  were  set  very  conservatively  in  order  to  capture  all  changes 
in  vibratory  patterns.  Since  this  is  an  emerging  technology,  B.F.  Goodrich  Corporation  wants  to 
ensure  that  their  sensors  do  not  inadvertently  miss  any  vibratory  indications  that  could  be  used  to 
indicate  impending  component  failure.  However,  the  conservative  setting  may  increase  the 
chance  of  false  alarms.  The  mean  value  of  Pc  will  be  varied  in  Chapter  5  in  order  to  explore 
sensitivity  of  the  simulation  results  to  its  value.  Furthermore  false  positives  (false  alarms)  will 
also  be  introduced  into  the  simulation  in  order  to  observe  their  impact  on  the  CBM+  process. 

The  Legacy  maintenance  schedule  of  the  UH-60  Blackhawk  70 1C  Engine 
Assembly  will  be  used  to  describe  the  specification  of  the  probabilities  of  the  Legacy  system 
discovering  a  previously  generated  DS  during  the  downtime  with  h  MMH,  Piih).  Although  the 
70 1C  Engine  Assembly  is  used  by  both  the  AH-64  Apache  and  the  UH-60  Blackhawk  the  two 
airframes  have  different  maintenance  schedules.  Listed  in  Table  8  below  is  the  maintenance 
schedule  for  the  UH-60  Blackhawk  70 1C  Engine  Assembly  maintenance  schedule  as  listed  in  the 
AMCOM  Proof  of  Principle  briefing  from  June  of  2005.  (Brown,  2005) 
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Table  8.  UH-60  Blackhawk  70 1C  Engine  Assembly  Maintenance  Regime 


Inspection 

Reference 

.  1 

Maint  Type 

Task  or  Paragraph 

MMH 

Pre-Flight 

TM  1-1520-237-CL 

PM 

Not  Applicable 

0.1 

Post-Flight 

TM  1-1520-237-CL 

PM 

Not  Applicable 

- 

Daily 

TM  1-1520-237-PMD 

PM 

6.17 

0.2 

40  Fir  Output  Shaft  Inspection 

TM  1-1520-237-23 

PM 

ch  1  sec  7-10.2 

0.4 

120  Fir  DS  Inspection 

TM  1-1520-237-23 

-18 

ch  1  sec  7-11.1 

3.4 

120  Hr  Clean  Engine  Compressor 

TM  1-2840-248-23 

-18 

1.158  thru  1.161 

1.6 

350  Hr 

TM  1-1520-237-PMI 

PM 

6.37,  6.43 

5.9 

700  Hr 

TM  1-1520-237-PMI 

PM 

6.37  -  6.48 

12.4 

It  is  evident  that  the  most  time  intensive  inspection  occurs  every  700  hours  and  it  requires 
12.4  maintenance  man-hours  (MMH).  12.4  MMH  is  the  hmax.  We  assume  the  probability  of 
detecting  a  DS  during  this  inspection  is  Pl( 700)=  Pc.  We  assume  that  the  amount  of  MMH 
expended  during  an  inspection,  /?,  is  an  indication  of  the  probability  of  discovering  a  DS  of 
Piih).  We  model  the  probability  of  detecting  a  DS  for  the  other  inspections  as  follows: 

1 .  Set  xhtwa  =  1  -  PL  (/?max )  for  the  known  most  MMH-intensive  inspection; 

i 

2.  Solve  for  x  =  (1  -  PL (hmax  ))'w  ; 

3.  The  probability  of  detecting  a  DS  during  an  inspection  lasting  h  hours  is 

h 

Pi(/?)  =  l_(l_Pi(/?ffiis))'w. 

Using  this  methodology  and  Table  8  the  following  values  are  computed  for  Pdh)  at 
differing  maintenance  inspection  intervals. 


Table  9,  UH-60  Blackhawk  701C  Engine  Assembly  Pl 


Inspection  Interval 

Maintenance  Man-Hours 

PlOO 

Baseline* 

0.3 

0.11 

40  Hour 

0.4 

0.14 

120  Hour 

5 

0.84 

350  Hour 

5.9 

0.89 

700  Hour 

12.4 

0.99 

*  Baseline  is  summed  total  of  Pre/PostFlight  and  Daily  inspection  times 
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This  provides  realistic  models  of  varying  levels  of  Pi{h)  that  can  be  input  into  the 
simulation.  The  generation  of  the  times  until  DS  discovery  for  both  processes  is  detailed  in 
Section  D  below. 

C.  ANALYTICAL  VERSUS  STOCHASTIC  RESULTS 

Results  obtained  from  the  simulation  and  the  analytical  formulas  are  displayed  in  Table 
10.  Examination  of  the  results  can  determine  whether  or  not  the  simulation  model  and  the 
analytical  model  results  are  comparable.  Since  DS  arrivals  drive  all  other  factors  in  this 
simulation  it  is  important  to  evaluate  the  number  of  DS  arrivals  during  the  course  of  1,250 
missions  (5,000  flight  hours).  The  expected  number  of  DSs  that  are  generated  during  1250 

missions  is  y5000^  =  1 1.42  where  y  =  5.4*10-6  and  8  =  1.71.  The  simulation  model  with  1,000 
replications  results  in  a  mean  number  of  DSs  generated  equal  to  1 1.34  with  a  95%  confidence 
interval  of  (1 1.14,  1 1.54).  Thus  the  generation  of  DSs  in  the  analytical  model  and  the  simulation 
are  in  good  statistical  agreement. 

Results  from  a  simulation  of  the  number  of  mission  aborts  during  500  missions  are 
displayed  in  Table  10.  Each  simulation  has  500  replications.  The  time  until  a  DS  is  discovered 
has  a  geometric  distribution  with  constant  probability  of  success  Pc.  The  time  until  a  DS  causes 
a  mission  abort  has  a  geometric  distribution  with  probability  of  success  Pa;  the  two  times  are 
assumed  independent.  The  mean  number  of  system  aborts  and  the  corresponding  expected 
number  of  system  aborts  obtained  from  the  analytical  model  equations  (1)  and  (2)  are  displayed. 
As  expected  the  analytical  results  fall  within  the  95%  confidence  interval  of  the  simulation 
results. 


Table  10.  Special  Case  Analytical  and  Simulation  Results 


Case 

Analytical 

Simulation  (95%  Cl) 

T 

o 

II 

Ci 

Tl 

> 

II 

5.99 

6.02  (5.87,6.17) 

Tl 

o 

II 

Ci 

Tl 

> 

II 

k> 

11.44 

11.60  (11.40,11.80) 

PL=  3,  Pa=.2 

19.50 

19.77  (19.49,20.05) 

D.  RANDOM  NUMBER  GENERATION  USING  CRYSTAL  BALL 

The  add-in  Crystal  Ball  version  5.5  is  used  as  the  random  number  generator  (RNG)  for 
this  simulation.  Crystal  Ball  version  5.5  is  chosen  to  provide  the  random  numbers  for  this 
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simulation  because  Microsoft  Excel  2003’ s  RNG  has  been  shown  to  have  insufficient  period 
length  and  an  incorrect  implementation  of  the  Wichmann-Hill  Algorithm.  (McCullough  and 
Wilson,  2005)  Crystal  Ball  is  considered  one  of  the  industry’s  leading  edge  Monte  Carlo 
simulation  add-ins  for  Excel.  It  uses  a  multiplicative  congruential  generator  that  has  a  stream  of 
231-1  pseudo  random  numbers  before  repeating.  The  iteration  formula  uses  the  multiplier 
62089911.  Crystal  Ball  produces  a  cycle  of  random  numbers  that  repeats  only  after  several 
billion  trials.  (Decisioneering,  2005) 

The  times  until  detection  of  DSs  are  generated  for  the  CBM+  process  by  drawing  a 
random  variable  from  an  exponential  distribution  with  a  mean  of  1  and  multiplying  that  value  by 

—  then  rounding  the  value  up  to  the  next  integer;  0C  is  equal  to  -  ln(l  -  Pc ) .  This  gives  the 

ec 

number  of  downtimes  until  the  DS  is  discovered.  If  the  downtime  is  one,  the  DS  is  discovered 
during  the  downtime  immediately  following  the  mission  during  which  it  was  generated.  The 
generation  for  0L(h)  is  based  on  the  maintenance  schedule  for  the  component.  For  example  if 
an  arduous  inspection  occurs  every  fifty  operating  hours  then  that  is  reflected  in  an  increase  in 
the  probability  a  DS  is  discovered,  Pdh),  and  subsequently  0L(h)  =  -ln(l-Pi  (/?)).  The  time 

until  detection  of  a  DS  for  the  Legacy  process  is  determined  by  generating  an  exponential 
random  variable  with  mean  1,  Y,  and  determining  the  smallest  n  such  that 
Y  <0L  (I\)  +  ...  +  0L  (/?„)  where  /?„  is  the  MMH  of  the  nth  inspection  after  generation  of  the  DS 

with  }\  being  the  MMH  for  the  inspection  following  the  mission  the  DS  is  generated  in.  A 

common  independent  exponential  random  variable  with  a  mean  of  1  is  used  to  generate  the  time 
until  DS  discovery  for  the  Legacy  process  and  for  the  CBM+  process.  The  operational  time  until 
a  DS  causes  a  mission  abort  is  generated  by  drawing  an  independent  random  variable  from  a 
Weibull  distribution.  Since  the  Weibull  distribution  is  a  continuous  distribution  the  value  is 
rounded  up  to  the  nearest  whole  number.  The  time  until  mission  abort  and  the  time  until  DS 
discovery  are  then  compared  and  whichever  event  occurs  first  is  the  event  that  happens;  the  other 
event  is  ignored.  If  the  two  discrete  times  are  equal,  the  mission  is  aborted.  N;  is  the  number  of 
DSs  that  originate  in  a  given  mission  (i).  The  simulation  is  designed  so  that  the  N;  is  identical 
for  both  the  Legacy  and  CBM+  processes.  Crystal  Ball  uses  the  method  of  inverse 
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transformation  to  generate  both  the  Exponential  and  Weibull  random  variables.  (Decisioneering, 
2005) 
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V.  DATA  ANALYSIS 


A.  VARYING  CBM+  SENSOR  EFFICIENCY 

One  of  the  measures  to  evaluate  a  component  for  selection  to  the  CBM+  process  is  the 
level  of  CBM+  sensor  efficiency.  In  particular  how  effective  must  a  CBM+  sensor  be  in 
discovering  DSs  to  result  in  a  smaller  number  of  aborted  missions  compared  to  that  of  the 
Legacy  process.  Varying  sensor  efficiency  is  defined  as  varying  the  mean  probability  of 
successfully  discovering  a  particular  DS,  Pc,  in  the  simulation.  Appendix  E  contains  tables 
displaying  the  statistical  summaries  of  the  simulation  output  for  the  Legacy  and  CBM+  processes 
as  the  CBM+  sensor  efficiency  mean  value  is  varied  from  0.99  to  0.1.  The  value  of  Pc  for  each 
generated  DS  is  determined  by  an  independent  draw  from  a  Beta  distribution.  Table  1 1  displays 
the  beta  parameter  values  used  in  the  simulation. 


Table  11.  Pc  Beta  Distribution  Parameters 


E(X)=P„ 

VARfXl 

Ob 

£b 

0.99 

0.00037711 

25 

0.25252525 

0.90 

0.00312741 

25 

2.77777777 

0.80 

0.00496124 

25 

6.25 

0.70 

0.00571984 

25 

10.71428571 

0.60 

0.00562500 

25 

16.66666666 

0.50 

0.00490196 

25 

25 

0.40 

0.00377953 

25 

37.5 

0.30 

0.00249012 

25 

58.33333333 

0.20 

0.00126984 

25 

100 

0.10 

0.00035857 

25 

225 

The  time  until  a  DS  causes  a  mission  abort  has  a  Weibull  distribution  with  mean  10  and 
shape  parameter  aw=L5.  The  number  of  replications  in  the  simulation  is  set  to  1000.  Although 
Pc  is  randomized  for  each  DS  for  the  CBM+  process,  the  values  of  Pl(Ii)  are  not  randomized  for 
the  Legacy  process  and  therefore  the  only  changes  in  the  Legacy  process  observed  between 
simulations  of  different  Pcs  are  due  to  the  variability  inherent  in  the  stochastic  nature  of  the 
simulation.  In  Appendix  E  the  statistical  summaries  of  the  simulation  output  are  displayed  and 
indicate  very  small  standard  errors  for  the  estimates  of  mean  number  of  mission  aborts  in  the 
following  graphs. 
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Figure  6. 


Figure  7. 


Mean  Number  of  Mission  Aborts  out  of  1,250  Missions  when  time  until  Mission  Aborts 
has  a  Weibull  distribution  with  mean  10  and  an.=  1.5 


Mean  Number  of  Mission  Aborts  out  of  1,250  Missions 
when  time  until  Mission  Aborts  has  a  Weibull  distribution 
with  mean  10  and  aw=1 .5 


Legacy  Process 
*—  CBM+  Process 


Mean  Repair  Times  for  1 ,250  Missions  when  time  until  Mission  Aborts  has  a  Weibull 
distribution  with  mean  10  and  air=  1.5 


Mean  Repair  Times  (MMH)  for  1 ,250  Missions  when  time 
until  Mission  Aborts  has  a  Weibull  distribution  with  mean  10 

and  aw=1-5 


Legacy  Process 


— ■—  CBM+  Process 


66 


As  shown  in  Figures  6  and  7  the  CBM+  Process  has  a  smaller  mean  number  of  system 
aborts  and  mean  repair  time  than  the  Legacy  Process  until  the  probability  of  successful  discovery 
of  a  DS  during  a  downtime  for  the  CBM+  sensor  is  degraded  to  approximately  12-15%.  This 
corresponds  roughly  with  the  average  of  PlUi,,)  (n=l,2,...  1,250)  where  h„  is  the  MMH  for  the 
Legacy  inspection  after  mission  n  that  is  determined  using  the  methodology  in  Chapter  4;  the 
baseline  Pith)  is  determined  using  the  sum  of  the  daily,  preflight,  and  postflight  inspection 
MMH  requirements.  Since  the  baseline  MMH  is  the  most  common  MMH  requirement,  the 
baseline  Pdh)  is  approximately  equal  to  the  average  of  Pi{h) s.  Abort  times  depend  upon  the  aw 
and  [iw  Weibull  distribution  parameters  selected  for  MA  as  described  in  Chapter  4.  In  the 
examples  listed  above  the  mean  value  selected  for  MA’s  Weibull  distribution  was  10  with  shape 
parameter  a.w=  1.5  and  scale  ^w=l  1 .077321 68.  This  means  that  on  average  it  requires  2.5 
missions  for  an  engine’s  diagnostic  symptom  to  become  a  mission  abort  situation;  the  variance  of 
the  time  until  mission  abort  is  46.8059.  This  value  was  selected  as  a  placeholder  and  does  not 
indicate  an  actual  estimation  of  engine  abort  occurrences.  However,  it  does  serve  to  illustrate  a 
generic  abort  behavior  as  Pc  is  varied  as  seen  in  the  previous  two  figures. 

B.  EXAMINING  MISSION  ABORT  TIME  VARIANCES 

In  this  section  we  explore  the  effect  the  variance  of  the  time  from  when  a  DS  is  generated 
until  it  causes  mission  abort  has  on  the  mean  number  of  mission  aborts.  The  mean  time  from 
when  a  DS  is  generated  until  it  causes  a  mission  abort  is  constant  for  all  cases  studied  in  this 
section.  The  parameters  of  the  Weibull  distribution  are  varied  to  obtain  different  variances.  In 
this  section  the  parameters  of  the  beta  distribution  used  to  generate  Pc  for  each  DS  are  a#=25  and 
/?#=. 25  giving  an  expected  value  of  Pc  equal  to  0.99.  The  density  functions  of  3  different 
Weibull  distributions  are  displayed  in  Figures  8,  9,  and  10: 

Figure  8.  CASE  #1  Weibull  Density  Function  (mean=10,  arv=. 5, 


Weibull  Density  Function  (mean=10,  a=.5,  (3=5) 


67 


The  Weibull  distribution  whose  density  function  is  displayed  in  Figure  8  has  expected 
value  equal  to  10  and  variance  equal  to  500. 

Figure  9.  CASE  #2  Weibull  Density  Function  (mean=10,  aw=  1,  /br=10) 


Weibull  Density  Function  (mean=10,  a=1,  p=10) 


The  Weibull  distribution  whose  density  function  is  displayed  in  Figure  9  has  expected 
value  equal  to  1 0  and  variance  equal  to  1 00.. 


Figure  10.  CASE  #3  Weibull  Density  Function  (mean=10,  ar*=1.5,/?rr=11.08) 


Weibull  Density  Function  (mean=10,  a=1.5,  p=11.0773217) 


The  Weibull  distribution  whose  density  function  is  displayed  in  Figure  10  has  expected 
value  equal  to  10  and  variance  equal  to  46.09. 

The  results  of  simulations  of  the  two  processes  utilizing  the  three  different  cases  of  the 
Weibull  distribution  of  the  time  from  DS  generation  until  it  causes  mission  abort  with  the  mean 
Pc  equal  to  0.99  appear  below.  Summary  statistics  of  the  simulation  output  appearing  in 
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Appendix  E  indicate  very  small  standard  errors  for  the  estimates  of  the  fraction  of  DSs  that  cause 
mission  abort.  The  fraction  of  DSs  that  cause  mission  aborts  is  defined  as  the  number  of  DSs 
that  actually  cause  a  mission  abort  divided  by  the  total  number  of  DSs  that  occurred  during  1,250 
missions.  This  divisor  is  the  sum  of  the  DSs  that  caused  mission  aborts  and  DSs  that  did  not 
cause  mission  aborts. 

CASE  #1  (Weibull  with  mean  equal  to  10,  variance  equal  to  500,  a^O.5;  fiw=5) 

The  fraction  of  DSs  that  causes  a  mission  abort  in  the  Legacy  process  is  0.64;  the 
fraction  of  DSs  that  causes  a  mission  abort  is  0.37  in  the  CBM+  process.  Recall  that  if  more  than 
one  DS  can  cause  a  mission  to  abort,  the  mission  is  only  aborted  once.  This  means  that  given 
these  conditions  under  the  Legacy  process  a  DS  will  result  in  a  mission  abort  64%  of  the  time, 
whereas,  under  the  CBM+  process  a  DS  will  result  in  a  mission  abort  on  37%  of  the  time. 

CASE  #2  (Weibull  with  mean  equal  to  10,  variance  equal  to  100,  arr=l;  fj;v=  10) 

The  fraction  of  DSs  that  causes  a  mission  abort  in  the  Legacy  process  is  0.47;  the 
fraction  of  DSs  that  causes  a  mission  abort  is  0.10  in  the  CBM+  process.  Recall  that  if  more  than 
one  DS  can  cause  a  mission  to  abort,  the  mission  is  only  aborted  once.  This  means  that  given 
these  conditions  under  the  Legacy  process  a  DS  will  result  in  a  mission  abort  47%  of  the  time, 
whereas,  under  the  CBM+  process  a  DS  will  result  in  a  mission  abort  on  10%  of  the  time. 

CASE  #3  (Weiboll  with  mean  equal  to  10,  variance  equal  to  46.09,  a^l.5; Pw=\  1.08) 

The  fraction  of  DSs  that  causes  a  mission  abort  in  the  Legacy  process  is  0.40;  the  fraction 
of  DSs  that  causes  mission  abort  is  0.03  in  the  CBM+  process.  Recall  that  if  more  than  one  DS 
can  cause  a  mission  to  abort,  the  mission  is  only  aborted  once.  This  means  that  given  these 
conditions  under  the  Legacy  process  a  DS  will  result  in  a  mission  abort  40%  of  the  time, 
whereas,  under  the  CBM+  process  a  DS  will  result  in  a  mission  abort  on  3%  of  the  time. 

Thus  the  number  of  mission  aborts  depends  on  more  than  just  the  mean  time  from  when  a 
DS  is  generated  until  it  causes  mission  abort.  The  variance  associated  with  the  mission  abort 
arrival  time  can  be  equally  influential.  The  CBM+  Process  which  monitors  every  mission  with  a 
high  mean  probability  Pc  of  discovering  DSs  is  most  effective  in  preventing  mission  aborts  in  all 
cases  but  does  best  when  the  variance  of  the  time  from  when  a  DS  is  generated  until  it  causes 
mission  abort  is  small. 
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C.  EXAMINING  MISSION  ABORT  TIME  VARIANCES  WITH  DEGRADED  Pc 

In  this  section,  the  mean  probability  an  existing  DS  is  discovered  during  a  downtime  (Pc) 
is  degraded  from  the  value  of  0.99  (aB= 25,  /</,■=.  25)  to  the  smaller  values  of  0.50  (aB= 25,  fiB=25) 
and  0.20  (aB=25 ,  0B=IOO)  in  order  to  study  the  effect  of  the  variance  of  the  time  from  when  a  DS 
is  generated  until  it  causes  a  mission  abort  with  a  less  effective  sensor.  The  statistical  standard 
errors  of  the  summary  statistics  of  the  simulation  output  are  displayed  in  Appendix  E.  The 
results  for  the  Legacy  Process  remain  as  stated  above  since  a  varying  Pc  does  not  affect  the 
Legacy  Process.  Lor  all  cases  the  Weibull  parameters  of  the  time  until  an  undiscovered  DS 
causes  a  mission  abort  ajv=l.5  and Pw=\  1 .08  are  held  constant. 

CASE  #1  (Weibull  with  mean  equal  to  10,  variance  equal  to  500,  an— 0.5;  fin— 5) 

This  case  possesses  the  largest  variance  and  results  in  the  probability  an  arriving  DS  will 
cause  a  mission  abort  for  the  CBM+  process  equal  to  0.45  (Pc—0.50)  /  0.57  (Pc— 0.20)  of  the 
time.  This  means  that  given  a  Pc  of  0.50  that  a  DS  will  result  in  a  mission  abort  45%  of  the  time. 
Given  a  Pc  of  0.20,  a  DS  will  result  in  a  mission  abort  57%  of  the  time. 

CASE  #2  (Weibull  with  mean  equal  to  10,  variance  equal  to  100,  aw=l;  fiw=l0) 

This  case  possesses  the  middle  variance  and  results  in  the  probability  an  arriving  DS  will 
cause  a  mission  abort  for  the  CBM+  process  equal  to  0.17  (/Y=0.50)  /  0.35  (Pc— 0.20)  of  the 
time.  This  means  that  given  a  Pc  of  0.50  that  a  DS  will  result  in  a  mission  abort  17%  of  the  time. 
Given  a  Pc  of  0.20,  a  DS  will  result  in  a  mission  abort  35%  of  the  time. 

CASE  #3  (Weibull  with  mean  equal  to  10,  variance  equal  to  46.09,  aw=l-5;  |3w=l  108) 

This  case  possesses  the  smallest  variance  and  results  in  the  probability  an  arriving  DS 
will  cause  a  mission  abort  for  the  CBM+  process  equal  to  0.09  (Pc= 0.50)  /  0.25  (Pc=0.20)  of  the 
time.  This  means  that  given  a  Pc  of  0.50  that  a  DS  will  result  in  a  mission  abort  9%  of  the  time. 
Given  a  Pc  of  0.20,  a  DS  will  result  in  a  mission  abort  25%  of  the  time. 

These  results  of  examining  Pc  at  both  0.50  and  0.20  are  consistent  with  the  conclusions 
drawn  from  examining  mission  abort  arrival  time  variance  when  Pc  is  0.99.  Furthermore,  it  is 
concluded  that  even  a  very  poor  performing  sensor  (i.e.  Pc= 0.20)  when  compared  against  the 
Legacy  process  reduces  the  percentage  of  diagnostic  symptoms  that  can  result  in  mission  aborts 
by  an  average  of  10%  regardless  of  the  variance  of  the  time  from  when  a  DS  is  generated  until  it 
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can  cause  a  mission  abort.  When  the  Pc  is  at  0.99  the  CBM+  Process  reduces  the  percentage  of 
diagnostic  symptoms  that  can  result  in  a  mission  aborts  by  an  average  of  30%-35%.  Legacy 
process  performance  data  used  in  the  comparisons  above  for  these  cases  is  displayed  in 
Appendix  E. 

D.  CBM+ FALSE  POSITIVES 

It  is  worthwhile  to  explore  the  consequences  of  the  CBM+  sensor  indicating  a  false 
positive.  Since  the  Army  currently  uses  only  passive  sensors  whose  measurements  are 
downloaded  at  the  conclusion  of  missions  a  false  positive  does  not  impact  the  mission  during 
which  it  occurs.  However,  a  false  positive  will  require  maintenance  personnel  to  expend 
maintenance  man-hours  in  order  to  determine  that  nothing  is  wrong  with  the  component.  The 
time  required  to  inspect  (repair)  and  correctly  diagnose  a  diagnostic  symptom  as  a  false  positive 
is  the  same  as  if  it  were  an  actual  diagnostic  symptom  recognized  immediately  after  the  mission 
during  which  it  arrived.  This  time  penalty  is  a  random  variable  drawn  from  an  exponential 
distribution  with  a  mean  of  3  hours.  False  positives  are  assumed  to  occur  according  to  a  Poisson 
process  independent  of  the  other  processes.  Table  12  displays  the  mean  number  of  false  positive 
arrivals  and  their  associated  mean  inspection  (repair)  times  resulting  from  a  simulation  with 
1,000  replications.  The  parameters  of  the  simulation  are:  «b=25,  /?#= 0.25  (Pc= 0.99);  aw=l.5, 
/?rr—  1 1  08  (mean  time  until  mission  abort  is  10  missions).  The  total  amount  of  repair  expended 
on  false  positives  that  appear  during  an  operational  time  /  is  a  compound  Poisson  process. 

A[total  repair  time]  =  £[ number  of  false  positives]  xE[ repair  time  per  false  positive] 

An  expanded  version  of  Table  12  appears  in  Appendix  E  annotating  standard  errors. 
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Table  12. 


Simulation  Results  for  mean  number  of  False  Positive  Arrivals  and  mean  Inspection 
(Repair)  Time  resulting  from  False  Positives  for  1,250  missions 


False 

Positive 

Rate 

CBM+  PROCESS  (NUMBER  OF 
FALSE  POSITIVE  ARRIVALS) 

0.9999 

FFP 

Mean: 

1238.57 

0.90 

FFP 

Mean: 

1115.29 

0.80 

FFP 

Mean: 

990.51 

0.70 

FFP 

Mean: 

866.70 

0.60 

FFP 

Mean: 

743.41 

0.50 

FFP 

Mean: 

620.23 

0.40 

FFP 

Mean: 

495.29 

0.30 

ZFP 

Mean: 

372.07 

0.20 

FFP 

Mean: 

247.05 

0.10 

FFP 

Mean: 

124.34 

0.0001 

FFP 

Mean: 

0.14 

False 

CBM+  PROCESS  (REPAIR 

Positive 

TIME  INCURRED  DUE  TO 

Rate 

FALSE  POSITIVE  ARRIVALS) 

0.9999 

J_FP  Time 

Mean: 

3717.41 

0.90 

yFP  Time 

Mean: 

3343.55 

0.80 

JFP  Time 

Mean: 

2969.66 

0.70 

IFP  Time 

Mean: 

2598.47 

0.60 

IFP  Time 

Mean: 

2228.17 

0.50 

IFP  Time 

Mean: 

1861.03 

0.40 

IFP  Time 

Mean: 

1490.28 

0.30 

JFP  Time 

Mean: 

1118.76 

0.20 

JFP  Time 

Mean: 

741 .94 

0.10 

7FP  Time 

Mean: 

371.62 

0.0001 

7FP  Time 

Mean: 

0.42 

Displayed  in  Table  13  are  the  analytical  results  for  the  number  of  false  positive  arrivals 
and  the  resulting  inspection  (repair)  time  for  1,250  missions. 
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Table  13.  Analytical  Results  for  mean  number  of  False  Positive  Arrivals  and  mean  Inspection 
(Repair)  Time  resulting  from  False  Positives  for  1,250  missions 


False 

Positive 

Rate 

CBM+  PROCESS  (NUMBER  OF 
FALSE  POSITIVE  ARRIVALS) 

0.9999 

XFP 

Mean: 

1249.88 

0.90 

XFP 

Mean: 

1125.00 

0.80 

XFP 

Mean: 

1000.00 

0.70 

XFP 

Mean: 

875.00 

0.60 

XFP 

Mean: 

750.00 

0.50 

XFP 

Mean: 

625.00 

0.40 

XFP 

Mean: 

500.00 

0.30 

XFP 

Mean: 

375.00 

0.20 

XFP 

Mean: 

250.00 

0.10 

XFP 

Mean: 

125.00 

0.0001 

XFP 

Mean: 

0.13 

False 

CBM+  PROCESS  (REPAIR 

Positive 

TIME  INCURRED  DUE  TO 

Rate 

FALSE  POSITIVE  ARRIVALS) 

0.9999 

XFP  Time 

Mean: 

3749.63 

0.90 

XFP  Time 

Mean: 

3375.00 

0.80 

XFP  Time 

Mean: 

3000.00 

0.70 

XFP  Time 

Mean: 

2625.00 

0.60 

XFP  Time 

Mean: 

2250.00 

0.50 

XFP  Time 

Mean: 

1875.00 

0.40 

XFP  Time 

Mean: 

1500.00 

0.30 

XFP  Time 

Mean: 

1125.00 

0.20 

XFP  Time 

Mean: 

750.00 

0.10 

XFP  Time 

Mean: 

375.00 

0.0001 

XFP  Time 

Mean: 

0.38 

As  expected  the  simulation  results  closely  match  the  analytical  results  and  produce  a 
linear  relationship  as  the  false  positive  rate  is  varied  from  0.9999  (false  positive  arriving  every 
mission)  to  0.0001  (false  positives  very  rarely  occurring). 

Assuming  that  the  Legacy  process  does  not  produce  a  false  positive  of  its  own  it  is  useful 
to  note  that  the  Legacy  process  incurs  approximately  75  hours  in  repair  time  every  1,250 
missions.  The  Legacy  process  expected  repair  time  includes  all  repair  times  including  repair  time 
resulting  from  mission  aborts.  Table  14  displays  the  mean  CBM+  Process  total  repair  time  (total 
repair  time  =  diagnostic  symptom  repair  time  +  false  positive  repair  time)  while  varying  the  false 
positive  rate  from  0.10  to  0.0001.  The  mean  time  until  mission  abort  is  10  missions  (atr=  1.5, 
jSw=l  1.08).  An  expanded  table  including  standard  errors  is  displayed  in  Appendix  E. 
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Table  14. 


CBM+  Process  mean  Total  Repair  Time  (including  False  Positive  Repair  Time)  for 
1,250  missions  with  aB=  25,  pB=. 25 


False 

CBM+  PROCESS  (TOTAL  REPAIR 

Positive 

TIME=DS  REPAIR  TIME+FALSE 

Rate 

POSITIVE  REPAIR  TIME) 

0.10 

7R+7FP  Time 

Mean: 

406.13 

0.09 

FR+5TP  Time 

Mean: 

369.59 

0.08 

7R+7FP  Time 

Mean: 

330.84 

0.07 

FR+FFP  Time 

Mean: 

291.50 

0.06 

7R+7FP  Time 

Mean: 

257.54 

0.05 

FR+FFP  Time 

Mean: 

221.93 

0.04 

7R+7FP  Time 

Mean: 

185.08 

0.03 

FR+FFP  Time 

Mean: 

146.10 

0.02 

FR+IFP  Time 

Mean: 

108.97 

0.01 

7R+7FP  Time 

Mean: 

71.91 

0.0001 

IR+IFP  Time 

Mean: 

36.06 

Figure  1 1  is  a  graphical  display  of  the  total  mean  repair  time  of  the  Legacy  and  CBM+ 

Process  as  the  false  positive  rate  is  varied  from  .05  to  .0001. 

Figure  11.  Mean  Total  Repair  Time  for  1,250  Missions  while  varying  the  False  Positive  rate  from 

.05  to  .0001 


Mean  Total  Repair  Time  for  1 ,250  Missions  while  varying 
the  False  Positive  rate  from  .05  to  .0001 


CBM+  Process  False  Positive 
Probability  of  Occurence 


Legacy  Process 
CBM+  Process 


Figure  1 1  illustrates  the  importance  of  limiting  the  number  of  false  positives  that  the 
CBM+  sensor  incurs.  This  simulation  was  run  with  a  mean  probability  of  DS  discovery  during  a 
downtime  equal  to  Pc= 0.99  (a  highly  accurate  sensor)  yet  when  false  positive  rates  begin  to 
increase  any  advantage  in  repair  time  provided  by  the  sensor  is  quickly  lost;  the  parameters  of 
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the  beta  distribution  for  Pc  are  aB= 25  and  j>u=25.  The  parameters  of  the  Weibull  distribution  are 
aw=  1.5  and  /ftr=l  1.08  (mean  time  until  mission  abort  is  10  missions). 

E.  LOGISTICAL  IMPLICATIONS  OP  VARYING  \\ 

It  is  possible  to  utilize  this  simulation  to  gain  insights  into  the  logistical  implications  of 
utilizing  the  CBM+  Process  versus  the  traditional  Legacy  Process.  The  following  is  a  general 
description  of  the  logistics  process.  An  assumption  made  is  that  once  a  DS  occurs  that  the  time 
remaining  until  mission  abort  is  known;  this  is  very  optimistic.  Another  assumption  is  that  there 
is  only  mission  per  day  and  that  a  mission  occurs  every  day.  This  is  differs  from  the  current 
implementation  of  CBM+  ;  once  a  component  shows  degraded  performance  it  is  removed;  the 
time  until  failure  is  not  known.  For  the  case  of  a  mean  time  until  a  replacement  component 
arrives  of  3  days  (assuming  one  mission  for  every  day)  the  following  process  is  used: 

1.  A  DS  is  detected. 

2.  If  remaining  time  until  a  mission  abort  is  greater  than  or  equal  to  the  time  to  fly  3 
missions  the  3  missions  will  be  flown.  The  following  events  occur: 

a.  A  replacement  component  is  ordered  and  arrives  before  there  is  a  mission  abort;  there  is  no 
time  with  aircraft  down  waiting  for  the  replacement  part  (Tqh)-  Since  one  mission  is  flown  per 
day,  the  time  until  the  replacement  component  arrives  is  three  days.  This  assumes  both  the 
number  of  missions  until  mission  abort  is  known  and  the  time  until  the  replacement  component 
arrives  is  constant  and  known.  This  fixed  time  is  an  approximate  of  the  prognostics  capability  of 
the  CBM+  sensor  and  can  be  modified  as  further  research  is  done  in  this  area. 

3.  If  the  remaining  time  until  a  mission  abort  is  less  than  the  time  to  fly  3  missions,  the 
aircraft  will  not  fly  until  the  failing  component  is  replaced.  A  Bernoulli  random  variable  is 
generated  with  probability  of  success  Poh (probability  component  is  on  hand): 

a.  If  a  replacement  component  is  on  hand,  there  is  no  aircraft  downtime  waiting  for  the 
replacement  component. 

b.  If  component  is  not  on  hand  an  exponential  time  with  mean  of  3  is  generated  to 
determine  T0h  (time  in  days  spent  awaiting  component). 

The  following  charts  illustrate  two  cases.  The  first  case  is  when  a  component  requires  a 
mean  of  3  days  to  arrive  once  ordered  and  the  second  case  is  when  a  component  requires  a  mean 
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of  10  days  to  arrive  once  ordered.  Through  examining  these  two  situations  conclusions  can  be 
drawn  concerning  the  effect  of  a  CBM+  sensor  on  the  component  ordering  time  requirements. 
When  a  DS  is  discovered,  the  number  of  future  missions  that  can  be  flown  before  a  mission  abort 
is  calculated.  One  mission  is  flown  per  day.  Thus  the  number  of  additional  missions  that  can  be 
flown  is  compared  to  the  mean  number  of  days  until  a  replacement  component  can  arrive.  If  the 
number  of  missions  that  can  be  flown  without  a  mission  abort  is  less  than  the  mean  number  of 
days  until  a  replacement  component  arrives,  then  an  independent  Bernoulli  random  variable  is 
generated  to  determine  if  the  replacement  component  is  on  hand.  If  the  component  is  not 
immediately  available  then  an  Exponential  distribution  is  used  to  determine  the  amount  of  time 
required  for  component  arrival.  These  random  numbers  are  independent  for  the  Legacy  and 
CBM+  processes.  Poh  will  be  used  to  represent  the  probability  that  a  replacement  component  is 
on  hand  at  the  location  of  the  unit  requiring  the  component;  when  a  DS  is  discovered  whether  or 
not  a  replacement  component  is  at  the  location  is  independent  from  DS  to  DS.  There  is  no 
randomization  of  the  probability  a  replacement  component  is  available.  The  value  of  Poh  will  be 
varied  from  .9999  to  .0001  in  order  to  compare  the  mean  number  of  days  awaiting  replacement 
components  for  both  the  CBM+  system  and  the  Legacy  system.  The  number  of  missions  is  1,250 
and  there  are  1,000  replications.  The  parameters  of  the  CBM+  process  are  o.b=25  and  /?b=. 25. 
The  parameters  of  the  Weibull  distribution  are  afp=1.5,  /hf=11.08  (the  mean  time  until  mission 
abort  is  10  missions).  Upon  the  occurrence  of  a  mission  abort  or  if  the  DS  is  discovered  but  it  is 
not  at  least  3  days  before  the  mission  abort  were  to  occur  (CASE  #1)  or  ten  days  before  the 
mission  abort  were  to  occur  (CASE  #2)  then  a  Bernoulli  random  variable  is  generated  to 
determine  whether  or  not  the  component  is  on  hand  (Poh)  and  if  the  component  is  not  on  hand 
then  an  exponential  distribution  with  mean  3  days  (CASE  #1)  or  10  days  (CASE  #2)  is  used  to 
determine  the  number  of  days  spent  awaiting  component  (Ton)-  Summary  statistics  of  the 
simulation  output  appear  in  Appendix  E.  The  standard  errors  of  the  mean  number  of  days 
expended  until  a  replacement  component  arrives  are  small  and  this  information  is  located  in 
Appendix  E  for  all  graphs  listed  in  the  remainder  of  Chapter  5. 

1.  3  Day  Mean  Component  Ordering  Time 

The  first  case  examines  three  different  Pc  levels  for  the  CBM+  Process.  The  first  Pc 
level  is  .99.  The  mean  number  of  days  expended  awaiting  replacement  components  as  the  Poh  is 
varied  from  .9999  to  .0001  are  displayed  in  Figure  12.  DSs  causing  mission  aborts  are  included. 
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Figure  12.  Mean  Number  of  Days  Expended  Awaiting  Replacement  Components  where  Pc=.99  and 

Mean  Time  until  replacement  component  arrives=3  days 


Mean  Number  of  Days  Expended  Awaiting  Replacement  Components  varied 
by  Probability  Replacement  Component  in  Stock  for  1,250  missions 
Probability  of  CBM+  Detection  =  .99 
Mean  Order  Time  (Exponential)  =  3  days 


Probability  Replacement  Component  in  Stock 


— *—  Legacy  Process 
CBM+  Process 


Figure  12  shows  that  if  the  remaining  time  until  component  failure  is  known  perfectly 
then  utilizing  the  CBM+  Process  and  having  no  the  components  on  hand  when  Pc  is  .99  is 
equivalent  to  keeping  approximately  enough  of  the  components  on  hand  under  the  Legacy 
Process  to  have  a  replacement  component  immediately  available  70%  of  the  time.  The  reason 
the  mean  time  expended  waiting  for  replacement  components  is  so  different  between  the  two 
processes  is  that  since  the  CBM+  process  discovers  DSs  so  much  earlier  on  average  than  the 
Legacy  process,  the  CBM+  process  is  able  to  order  the  component  before  the  component  fails. 
The  Legacy  process  does  not  have  this  advantage  since  it  will  either  not  detect  until  a  mission 
abort  or  once  a  DS  is  discovered  it  is  too  late  to  order  the  component  before  the  component  fails. 
Figures  13  and  14  display  the  mean  delay  time  until  replacement  components  arrive,  for  Pc  equal 
to  0.50  (respectively  0.20);  if  the  remaining  time  until  component  failure  is  known  perfectly, 
then  the  result  for  the  case  the  CBM+  process  has  no  components  on  hand  is  the  same  that  for  the 
Legacy  process  that  has  enough  components  at  the  location  to  replace  60%  (respectively  30%) 
of  the  discovered  DSs  at  their  time  of  discovery. 

Figure  13.  Mean  Number  of  Days  Expended  Awaiting  Replacement  Components  where  Pc—  .50  and 

Mean  Order  Time=3  days 
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Figure  14.  Mean  Number  of  Days  Expended  Awaiting  Replacement  Components  where  Pc— .20  and 

Mean  Order  Time=3  days 
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2.  10  Day  Mean  Component  Ordering  Time 

If  the  mean  time  of  the  exponential  ordering  time  is  increased  to  10  days  this  will  provide 
insight  into  the  advantage  of  utilizing  the  CBM+  Process  for  a  component  that  is  not  as  readily 
available  in  the  Army’s  logistical  system. 


A  review  of  Figures  15,  16,  and  17  show  that  if  the  remaining  time  to  component  failure 
is  known  when  the  Pc  is  .99  /  .50  /  .20  and  no  components  are  kept  on  hand  that  the  equivalent 
mean  wait  time  for  ordered  components  is  the  same  for  keeping  enough  on  hand  under  the 


Legacy  Process  to  have  this  component  available  30%  /  20%  /  10%  of  the  time  respectively. 
This  suggests  that  if  a  component  is  to  be  chosen  for  entry  in  the  CBM+  Process  it  is  more 
beneficial  to  choose  a  component  that  can  be  shipped  faster  and  not  one  that  requires  longer  to 
arrive  to  the  organization  requiring  the  replacement  component.  The  advantage  that  the  CBM+ 
Process  delivers  is  that  it  gives  the  maintainer  advance  warning  that  a  component  will  fail  in  the 
near  future.  If  the  component  can  be  ordered  when  a  diagnostic  symptom  is  recognized  and  sent 
to  the  organization  requiring  the  component  before  the  component  causes  mission  abort  then 
great  savings  in  on  hand  stockage  requirements  can  be  realized;  this  assumes  that  the  remaining 
time  until  mission  abort  can  be  predicted  with  accuracy.  If  the  aircraft  is  grounded  until 
replacement  parts  become  available  then  the  CBM  process  is  comparable  to  the  Legacy  process. 
However,  if  the  component  requires  a  longer  time  to  arrive  then  this  advantage  that  the  CBM+ 
Process  possesses  over  the  Legacy  Process  still  exists  but  is  diminished. 


Figure  15.  Mean  Number  of  Days  Expended  Awaiting  Replacement  Components  where  Pc=  .99  and 

Mean  Order  Time=10  days 


Mean  Number  of  Days  Expended  Awaiting  Replacement 
Components  varied  by  Probability  Replacement  Component  in 
Stock  for  1,250  missions 
Probability  of  CBM+  Detection  =  .99 
Mean  Order  Time  (Exponential)  =  10  days 
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Figure  16.  Mean  Number  of  Days  Expended  Awaiting  Replacement  Components  where  Pc=. 50  and 

Mean  Order  Time=10  days 


Mean  Number  of  Days  Expended  Awaiting  Replacement  Components 
varied  by  Probability  Replacement  Component  in  Stock  for  1,250 

missions 

Probability  of  CBM+  Detection  =  .50 
Mean  Order  Time  (Exponential)  =  10  days 


Probability  Replacement  Component  in  Stock 


— *—  Legacy  Process 
— CBM+  Process 


Figure  17.  Mean  Number  of  Days  Expended  Awaiting  Replacement  Components  where  Pc=. 20  and 

Mean  Order  Time=10  days 


Mean  Number  of  Days  Expended  Awaiting  Replacement  Components  varied 
by  Probability  Replacement  Component  in  Stock  for  1,250  missions 
Probability  of  CBM+  Detection  =  .20 
Mean  Order  Time  (Exponential)  =  10  days 


— *—  Legacy  Process 
— ■—  CBM+  Process 
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VI.  RESULTS  AND  CONCLUSIONS 


A.  APPLICATIONS 

A  model  of  the  Legacy  maintenance/repair  process  and  the  CBM+  maintenance  /repair 
process  has  been  presented.  The  model  assumes  that  prior  to  a  component  failure,  a  measurable 
diagnostic  symptom  (DS)  appears.  Once  a  DS  is  generated,  it  remains  measurable  and  can  be 
detected  by  a  CBM+  sensor  or  by  physical  inspection  (Legacy  process).  The  model  output 
includes  the  number  of  missions  that  are  aborted  and  the  repair  time  incurred  by  component 
failures  and  false  positives.  By  comparing  two  or  more  different  components  it  is  possible  to 
determine  which  component  will  produce  more  favorable  results  in  terms  of  mission  abort  rates 
and  repair  time  expenditures  by  introducing  it  into  the  CBM+  process.  Furthermore,  patterns  and 
behaviors  can  be  observed  as  conditions  vary  thereby  providing  insight  and  information  to  be 
used  by  the  decision-maker. 

The  following  factors  are  influential  to  the  successful  introduction  of  a  component  in  a 
CBM+  program: 

1.  Since  the  CBM+  sensor  is  continually  monitoring  a  component,  the  sensor  doesn’t 
have  to  have  an  extremely  high  level  of  probability  of  detection  of  diagnostic  symptoms;  this 
result  assumes  that  the  probability  of  detection  of  a  DS  is  independent  from  mission  to  mission. 
It  also  assumes  that  once  a  DS  has  occurred  it  remains  detectable;  that  is  measurable  evidence  of 
the  DS  is  not  intermittent.  Simply  by  providing  a  level  of  detection  for  every  mission  that 
exceeds  the  baseline  (the  daily  pre-flight  and  post-flight  inspections)  probability  of  detection 
provided  by  the  Legacy  Process  the  CBM+  Process  will  show  a  substantial  increase  in  the 
maintainer’s  ability  to  recognize  and  mitigate  impending  mission  aborts.  However,  this 
advantage  will  decrease  if  the  sensor  produces  false  alarms. 

2.  When  selecting  a  component  for  entry  in  the  CBM+  Process  more  than  just  a 
comparison  of  the  mean  times  between  the  arrival  of  a  diagnostic  symptom  until  development  of 
a  mission  abort  are  required.  The  variance  of  the  mean  time  until  mission  abort  given  that  a 
diagnostic  symptom  has  occurred  is  equally  important.  A  CBM+  sensor  may  be  less  effective  if 
the  time  from  when  a  DS  is  generated  until  it  causes  a  mission  abort  has  a  high  variance.  This 
finding  remains  true  for  degraded  levels  of  mean  probability  a  DS  is  discovered,  Pc.  Data 
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concerning  the  time  from  when  a  DS  is  generated  until  component  failure  would  be  very 
informative  in  judging  whether  to  include  a  component  in  the  CBM+  process. 

3.  Whereas  a  component’s  CBM+  sensor  doesn’t  need  an  extremely  high  probability  of 
detection  that  a  DS  has  occurred,  it  is  extremely  important  that  false  positives  be  kept  to 
extremely  low  levels.  Otherwise,  the  advantage  of  continual  inspection  using  sensors  begins  to 
work  against  the  CBM+  Process.  If  a  component  whose  CBM+  sensor  provides  a  fair  number  of 
false  positives  (i.e.  5%  of  the  missions  result  in  a  false  positive)  the  time  spent  by  maintenance 
personnel  confirming  that  the  DS  was  in  fact  a  false  positive  quickly  overshadows  any  gains  in 
actual  repair  times. 

To  reiterate  some  factors  to  consider  when  considering  introducing  a  component  to  the 
CBM+  process  from  Chapter  3: 

1.  How  often  does  the  component  fail  during  active  flight  hours? 

It  is  preferable  to  introduce  a  component  that  fails  often  versus  components  that  rarely 
fail. 

2.  What  are  the  consequences  of  a  component  failing? 

It  is  preferable  to  introduce  a  component  that  possesses  severe  consequences  for  failure 
versus  a  component  that  is  inconsequential  in  its  failure. 

3.  What  is  the  difficulty  level  of  the  inspection  that  would  be  alleviated  by  CBM+? 

If  an  inspection  is  extremely  difficult  and  resource  intensive  to  perform,  it  would  be 
preferable  to  have  this  inspection  replaced  by  CBM+  versus  an  inspection  that  is  simple  and  easy 
to  perform. 

4.  Is  a  CBM+  sensor  feasible  for  a  component? 

If  measurements  cannot  be  taken  to  assess  the  degradation  of  the  component  before  it 
fails,  i.e.  no  useful  DS,  then  that  component  would  not  be  a  good  CBM+  candidate. 
Furthermore,  the  time  from  when  the  occurrence  of  measurable  evidence  of  impending  failure 
until  failure  should  be  long  enough  to  detect  the  impending  failure  and  take  maintenance  action. 
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5.  What  is  the  reliability  of  the  CBM+  sensor,  the  difficulty  of  inspecting  the  CBM+ 
sensor,  and  the  effect  of  the  CBM+  sensor  on  performance  of  the  component  being  monitored? 
Is  there  a  CBM+  sensor  currently  developed  for  a  component?  What  is  the  cost  of  the  sensor? 

B.  RECOMMENDATIONS  FOR  FUTURE  STUDY 

Models  for  the  following  random  variables  are  selected  as  placeholders.  Collection  and 
analysis  of  data  are  needed  to  provide  more  appropriate  models. 

1.  Ma:  (time  from  when  a  DS  occurs  until  it  causes  a  mission  abort)  A  Weibull 
distribution  with  a  mean  of  10  hours  was  selected.  The  distributional  form  is  conjectural. 
Experiments  need  to  be  designed  and  conducted  to  collect  data  concerning  the  time  from  when  a 
DS  is  generated  by  a  component  until  the  DS  causes  the  component  to  fail.  It  is  also  important  to 
collect  data  on  the  time  from  when  a  DS  is  generated  until  the  sensor  detects  the  DS. 

2.  R0:  (initial  repair  time)  An  Exponential  distribution  with  a  mean  of  3  hours  was 
selected.  This  provided  the  initial  mean  repair  time  incurred  upon  a  DS  discovery. 

3.  R| :  (subsequent  fixed  repair  time)  A  fixed  value  of  3  hours  was  chosen.  This  provided 
for  an  additional  repair  time  if  the  DS  was  not  discovered  after  the  first  mission  in  which  it 
arrived.  This  value  was  only  applied  once.  For  example,  Ri  was  the  same  if  the  DS  was 
discovered  after  the  second  mission  it  appeared  or  the  tenth  mission  after  it  appeared. 

4.  Ao:  (initial  repair  time  resulting  from  a  mission  abort)  An  Exponential  distribution 
with  a  mean  of  5  hours  was  selected.  This  provided  mean  repair  time  incurred  upon  a  mission 
abort. 

5.  Ay  (subsequent  fixed  repair  time  resulting  from  a  mission  abort)  A  fixed  value  of  5 
hours  was  chosen.  This  provided  for  an  additional  repair  time  if  the  DS  that  caused  the  mission 
abort  was  not  discovered  after  the  first  mission  in  which  it  arrived.  This  value  was  only  applied 
once.  For  example,  Ai  was  the  same  if  the  DS  was  discovered  after  the  second  mission  it 
appeared  or  the  tenth  mission  after  it  appeared. 

It  is  also  important  to  conduct  a  study  of  the  reliability  and  maintainability  of  the  CBM+  sensors. 
It  is  envisioned  that  CBM+  sensors  will  eliminate  the  need  for  scheduled  inspection  and 
maintenance.  A  sensor  may  be  able  to  detect  the  occurrence  of  a  DS;  however,  if  the  sensor  is 
experiences  failure  often  and  is  difficult  to  inspect  and  maintain,  the  effectiveness  of  the  sensor 
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will  be  diminished.  In  addition  it  is  important  to  study  the  ability  of  the  logistics  process  to 
respond  to  the  sensor  measurements.  If  it  takes  a  long  time  to  analyze  sensor  measurements 
and/or  obtain  replacement  components,  then  the  attractiveness  of  introducing  a  component  into 
CBM  is  lessened.  The  cost  of  the  sensor  also  needs  to  be  considered. 
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EXHIBIT  2,  APPENDIX  A  AH-64/UH-60  701C  ENGINE  DATA 


SN 

EI_SN 

MODEL 

Date 

REP_NUM 

CEN 

LIFE 

TSN 

F_TYPE 

FCODE 

Failure 

FAMILY 

PREV 

r_FC 

GEE761067 

9126370 

MH-60K 

5/17/2001 

1 

0 

944 

944 

C1_REMOVAL 

537 

LOW  POWER  OR  TORQUE 

Other 

GEE761067 

8926194 

MH-60K 

2/19/2002 

2 

0 

231 

1178 

C1_REMOVAL 

537 

LOW  POWER  OR  TORQUE 

Other 

537 

GEE761067 

9426547 

MH-60L 

1/8/2003 

3 

0 

246 

C1_REMOVAL 

LOW  POWER  OR  TORQUE 

Other 

537 

GEE761067 

9026285 

MH-60L 

3/9/2004 

4 

0 

285 

1712 

C1_REMOVAL 

LOW  POWER  OR  TORQUE 

Other 

537 

GEE761067 

9026285 

MH-60L 

8/5/2005 

4 

0 

372 

2085 

C1_REMOVAL 

374 

INTERNAL  FAILURE 

Assembly 

537 

GEE761 1 58 

9026295 

UH-60L 

10/11/2001 

1 

0 

1873 

1873 

LOW  POWER  OR  TORQUE 

Other 

GEE761 1 58 

9226426 

UH-60L 

9/26/2002 

2 

0 

256 

2129 

INTERNAL  FAILURE 

Assembly 

537 

GEE761 1 58 

8900227 

AH-64A 

3/15/2003 

3 

0 

245 

2374 

C1_REMOVAL 

585 

SHEARED 

Op/FOD 

374 

GEE761 1 58 

9626689 

UH-60L 

9/10/2003 

4 

1 

1034 

2376 

INSTALLED 

799 

NO  DEFECT-SERVICEABLE 

NoDefect 

585 

GEE761259 

9126329 

UH-60L 

2/14/2001 

1 

0 

1573 

1573 

C1_REMOVAL 

790 

OUT  OF  ADJUSTMT.OUT 

Tol/Bal 

GEE761259 

8900235 

AH-64A 

3/11/2003 

2 

0 

412 

1986 

C1_REMOVAL 

372 

METAL  ON  MAGNETIC 

790 

GEE761259 

UNKNOWN 

2/10/2004 

3 

0 

249 

2235 

C3C2ACT 

20 

WORN  EXCESSIVELY 

Rem 

372 

GEE761259 

9000482 

AH-64A 

9/15/2004 

4 

1 

339.8 

2349 

INSTALLED 

799 

NO  DEFECT-SERVICEABLE 

NoDefect 

0 

GEE761705 

9705043 

AH-64D 

2/22/2002 

1 

0 

1729 

1729 

BEYOND  SPECIFIED 

Tol/Bal 

GEE761705 

UNKNOWN 

11/29/2003 

2 

0 

254 

1988 

LOW  COMPRESSION 

Rem 

705 

GEE761705 

UNKNOWN 

8/19/2004 

3 

0 

97 

2085 

C3C2ACT 

855 

HEAT  DAMAGE 

Rem 

0 

GEE761705 

5201 

AH-64D 

8/19/2004 

4 

1 

305.8 

2085 

INSTALLED 

799 

NO  DEFECT-SERVICEABLE 

NoDefect 

0 

GEE761735 

9000294 

AH-64A 

12/11/2001 

1 

0 

1769 

1769 

C1_REMOVAL 

374 

INTERNAL  FAILURE 

Assembly 

GEE761735 

9000294 

AH-64A 

7/1 8/2003 

2 

0 

393 

2163 

C1_REMOVAL 

329 

STARTING  STALL 

374 

UNKNOWN 

2/4/2005 

3 

0 

165 

2330 

C3C2ACT 

230 

DIRTY 

Rem 

329 

GEE761735 

9926829 

UH-60L 

2/8/2005 

4 

1 

106.5 

2330 

INSTALLED 

799 

NO  DEFECT-SERVICEABLE 

NoDefect 

0 

GEE761745 

9000306 

AH-64A 

1 0/24/2002 

1 

0 

1812 

1812 

C1_REMOVAL 

519 

SURGED 

GEE761745 

UNKNOWN 

8/11/2003 

2 

0 

106 

1920 

C3C2ACT 

537 

LOW  POWER  OR  TORQUE 

Rem 

519 

GEE761745 

UNKNOWN 

1/24/2004 

3 

0 

125 

2045 

C3C2ACT 

230 

DIRTY 

Rem 

0 

GEE761745 

9326486 

UH-60L 

3/2/2004 

4 

0 

5 

2050 

C1_REMOVAL 

537 

LOW  POWER  OR  TORQUE 

Other 

0 

GEE761745 

9326525 

UH-60L 

2/1/2005 

4 

0 

284 

2336 

C1_REMOVAL 

374 

INTERNAL  FAILURE 

Assembly 

537 

GEE761745 

9626678 

UH-60L 

9/10/2005 

4 

1 

136.5 

2340 

INSTALLED 

799 

NO  DEFECT-SERVICEABLE 

NoDefect 

374 

GEE761816 

9126373 

MH-60K 

9/6/2002 

1 

0 

2055 

C1_REMOVAL 

LOW  POWER  OR  TORQUE 

Other 

GEE761816 

9026293 

UH-60L 

5/22/2003 

2 

0 

129 

C1_REMOVAL 

LOW  POWER  OR  TORQUE 

Other 

537 

GEE761816 

9126363 

MH-60L 

11/24/2003 

3 

0 

144 

2329 

C1_REMOVAL 

537 

LOW  POWER  OR  TORQUE 

Other 

537 

GEE761816 

9126360 

MH-60L 

6/5/2004 

4 

0 

61 

2390 

C1_REMOVAL 

537 

LOW  POWER  OR  TORQUE 

Other 

537 

GEE761816 

8926184 

MH-60L 

5/9/2005 

4 

0 

236 

2626 

C1_REMOVAL 

537 

LOW  POWER  OR  TORQUE 

Other 

537 

GEE761816 

8926184 

MH-60L 

6/8/2005 

4 

1 

118.7 

2626 

INSTALLED 

799 

NO  DEFECT-SERVICEABLE 

NoDefect 

537 

GEE762055 

9126379 

MH-60K 

4/22/2002 

1 

0 

2027 

2027 

C1_REMOVAL 

537 

LOW  POWER  OR  TORQUE 

Other 

GEE762055 

9126379 

MH-60K 

3/24/2003 

2 

0 

239 

2267 

C1_REMOVAL 

537 

LOW  POWER  OR  TORQUE 

Other 

537 

GEE762055 

UNKNOWN 

9/17/2005 

3 

0 

329 

2841 

C3C2ACT 

537 

LOW  POWER  OR  TORQUE 

Rem 

537 

GEE762055 

305400 

AH-64D 

1/3/2006 

4 

1 

2838.9 

2841 

INSTALLED 

799 

NO  DEFECT-SERVICEABLE 

NoDefect 

0 

GEE762287 

UNKNOWN 

1/8/2001 

1 

0 

1379 

1379 

C3C2ACT 

537 

LOW  POWER  OR  TORQUE 

Rem 

GEE762287 

9826825 

UH-60L 

8/20/2002 

2 

0 

487 

1866 

C1_REMOVAL 

537 

LOW  POWER  OR  TORQUE 

Other 

0 

GEE762287 

8926182 

UH-60L 

9/10/2003 

3 

0 

172 

2042 

C1_REMOVAL 

180 

CLOGGED 

537 

GEE762287 

9026294 

UH-60L 

5/12/2004 

4 

0 

223 

2265 

C1_REMOVAL 

537 

LOW  POWER  OR  TORQUE 

Other 

180 

GEE762287 

UNKN@L2 

UNKNOWN 

1/4/2005 

4 

1 

250.6 

2267 

INSTALLED 

799 

NO  DEFECT-SERVICEABLE 

NoDefect 

537 

GEE762385 

9326503 

UH-60L 

9/25/2002 

1 

0 

1549 

1549 

C1_REMOVAL 

290 

FAILS 

HeaterBk 

GEE762385 

9426583 

UH-60L 

2/2/2004 

2 

0 

50 

1599 

C1_REMOVAL 

70 

BROKEN 

Broken 

290 

GEE762385 

9426558 

UH-60L 

2/18/2005 

3 

0 

346 

1945 

C1_REMOVAL 

307 

OIL  LEAK 

70 

GEE762385 

9426587 

UH-60L 

7/27/2005 

4 

1 

283.1 

1945 

OIL  LEAK 

307 

GEE762436 

UNKNOWN 

2/16/2001 

1 

0 

1413 

1413 

LEAKING  (LIQUID) 

Rem 
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GEE762436 
GEE762436 
GEE762436 
GEE762440 
GEE762440 
GEE762440 
GEE762440 
GEE762523 
GEE762523 
GEE762523 
GEE762523 
GEE762606 
GEE762606 
GEE762606 
GEE762606 
GEE762993 
GEE762993 
GEE762993 
GEE762993 
GEE763028 
GEE763028 
GEE763028 
GEE763028 
GEE763102 
GEE7631 02 
GEE7631 02 
GEE7631 02 
GEE763260 
GEE763260 
GEE763260 
GEE763260 
GEE763260 
GEE763301 
GEE763301 
GEE763301 
GEE763301 
GEE763301 
GEE763301 
GEE763383 
GEE763383 
GEE763383 
GEE763383 
GEE763401 
GEE763401 
GEE763401 
GEE763401 
GEE763401 


305368 


205319 


9226452 

205295 

9026250 

9226452 


9626685 

9905114 


9426561 


9426561 

9526597 


9426552 


9705046 

9705046 


9026269 


9626674 

9626682 


9626676 


9705041 


9705041 


305347 


9826805 


9826805 

9826805 


9426587 


9605015 

9826823 

9826805 


9826826 


9026294 

8926182 


9526603 

9426558 


9926830 

9226448 

9126332 

9426575 

9426569 


9426568 


9426577 

9526649 


AH-64D 

5/5/2005 

AH-64D 

5/17/2005 

UNKNOWN 

11/13/2002 

UH-60L 

10/28/2003 

AH-64D 

11/17/2004 

UH-60L 

11/28/2005 

UNKNOWN 

12/9/2002 

UH-60L 

6/6/2003 

UH-60L 

11/3/2004 

AH-64D 

9/15/2005 

UH-60L 

9/9/2003 

UH-60L 

12/9/2003 

UH-60L 

3/1/2004 

UH-60L 

7/13/2004 

UNKNOWN 

11/22/2002 

UNKNOWN 

10/23/2003 

AH-64D 

4/30/2004 

AH-64D 

5/20/2004 

UH-60L 

2/2/2002 

UH-60L 

10/26/2004 

UH-60L 

12/27/2004 

UH-60L 

9/23/2005 

AH-64D 

6/29/2005 

AH-64D 

6/29/2005 

UNKNOWN 

9/11/2005 

AH-64D 

12/31/2005 

UH-60L 

2/23/2001 

UH-60L 

2/21/2002 

UH-60L 

6/16/2003 

UH-60L 

10/20/2003 

AH-64D 

1/10/2005 

UH-60L 

5/8/2003 

UH-60L 

8/30/2003 

UH-60L 

1/15/2004 

UH-60L 

5/27/2004 

UH-60L 

12/30/2004 

UH-60L 

4/19/2005 

UH-60L 

3/8/2003 

UH-60L 

8/25/2004 

UH-60L 

11/16/2004 

UH-60L 

8/10/2005 

UH-60L 

12/1/2001 

UH-60L 

3/27/2003 

UH-60L 

9/11/2003 

UH-60L 

11/7/2003 

UH-60L 

7/14/2005 

0  514 


0  102 


241.2 


0  1488 

”6  236 

”6  251 

1  23.9 

”0  1432 

”6  124 


0  352 

1  209.6 


0  2115 


0  14 

”0  5 


1  576.8 

”6  1082 
”6  149 


0  135 

1  1159.6 


0  956 


1  573.5 

”6  983 

”6  163 


0  134 


0  513 

”o  r~ 

1  104.2 

”0  422~" 

”0  266” 


C1_REMOVAL 


C1_REMOVAL 


INSTALLED 


C3C2ACT 

C1_REMOVAL 

C1_REMOVAL 

INSTALLED 

C3C2ACT 

C1_REMOVAL 


C1_REMOVAL 

INSTALLED- 


C1_REMOVAL 


C1_REMOVAL 

C1_REMOVAL 


INSTALLED 

C3C2ACT 

C3C2ACT 


C1_REMOVAL 

INSTALLED- 


C1_REMOVAL 


C1_REMOVAL 


C1_REMOVAL 


C1_REMOVAL 


C1_REMOVAL 


C1_REMOVAL 


C3C2ACT 


INSTALLED 


C1_REMOVAL 


C1_REMOVAL 


C1_REMOVAL 


C1_REMOVAL 


INSTALLED 

C1_REMOVAL 

C1_REMOVAL 


C1_REMOVAL 


C1_RE 

C1_RE 


INSTALLED 

C1_REMOVAL 


C1_REMOVAL 

C1_REMOVAL 

INSTALLED 

C1_REMOVAL 

C1_REMOVAL 


C1_REMOVAL 


C1_RE 
Cl  RE 


537  LOW  POWER  OR  TORQUE 


537  LOW  POWER  OR  TORQUE 


799  NO  DEFECT-SERVICEABLE 


537  LOW  POWER  OR  TORQUE 
537  LOW  POWER  OR  TORQUE 
”374  INTERNAL  FAILURE 

799  NO  DEFECT-SERVICEABLE 
537  LOW  POWER  OR  TORQUE 
181  LOW  COMPRESSION 


381  LEAKING  (LIQUID) 

799  NO  DEFECT-SERVICEABLE 


537  LOW  POWER  OR  TORQUE 


'  LOW  POWER  OR  TORQUE 
~  LOW  POWER  OR  TORQUE 


799  NO  DEFECT-SERVICEABLE 
537  LOW  POWER  OR  TORQUE 
181  LOW  COMPRESSION 


537  LOW  POWER  OR  TORQUE 
799  NO  DEFECT-SERVICEABLE 


307  OIL  LEAK 


315  RPM 


381  LEAKING  (LIQUID) 


537  LOW  POWER  OR  TORQUE 


537  LOW  POWER  OR  TORQUE 


537  LOW  POWER  OR  TORQUE 


537  LOW  POWER  OR  TORQUE 


799  NO  DEFECT-SERVICEABLE 


374  INTERNAL  FAILURE 


537  LOW  POWER  OR  TORQUE 


855  HEAT  DAMAGE 


537  LOW  POWER  OR  TORQUE 


799  NO  DEFECT-SERVICEABLE 
”290  FAILS 

"~20  WORN  EXCESSIVELY 


374  INTERNAL  FAILURE 


LOOSE  BOLTS,  NUTS, 
LEAKING  (LIQUID) 


799  NO  DEFECT-SERVICEABLE 
537  LOW  POWER  OR  TORQUE 


513  STALLS,  COMPRESSOR 

”513  STALLS,  COMPRESSOR 

799  NO  DEFECT-SERVICEABLE 
”381  LEAKING  (LIQUID) 

”381  LEAKING  (LIQUID) 


381  LEAKING  (LIQUID) 


OIL  LEAK 

;  STALLS.  COMPRESSOR 


Other 


Other 


NoDefect 


Rem 

Other 

Assembly 

NoDefect 

Rem 

Debond 


AirLeak 

NoDefect 


Other 


Other 

Other 


NoDefect 

Rem 

Rem 


Other 

NoDefect 


AirLeak 


Other 


Other 


Other 


Rem 


NoDefect 


Assembly 


Other 


NoDefect 

HeaterBk 

Erosion 


Assembly 


Assembly 

AirLeak 


NoDefect 

Other 


NoDefect 

AirLeak 

AirLeak 


AirLeak 
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EXHIBIT  2,  APPENDIX  B  ESTIMATED  INTENSITY  FUNCTIONS  USING  ENGINES 

WITH  3  OR  MORE  OBSERVED  FAILURES 

The  table  and  figure  in  this  Appendix  are  taken  from  Professor  Patricia  A.  Jacobs’  “A 


Nonhomogeneous  Poisson  process  model  (NHPP)  for  engine  data”  written  27  March  2006. 


Estimated  intensity  function  for  engines  with  at  least  3  recorded  failures 

Age  interval 

Number  of  failures 

(number  of  engines) 

Estimated  failure  rate 

0 

500 

2 

(21) 

0.00019 

501 

1000 

10 

(20) 

0.001 

1001 

1500 

19 

(20) 

0.0019 

1501 

2000 

21 

(18) 

0.0023 

2001 

2500 

16 

(9) 

0.0036 

2501 

3000 

2 

(2) 

0.002 
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EXHIBIT  2,  APPENDIX  C  ESTIMATION  OF  y  AND  5 


Let  |  A (/) ;/  >  0]  be  a  NHPP  with  mean  value  function  A(/)  =  ytd  having  intensity  function 

/.(/)  =  yStS~l .  There  are  K  systems.  The  kth  system  has  nk  observed  times  of  failure;  the  ith 

failure  occurs  at  time  tjk .  The  kth  system  is  observed  for  a  time  Tk  .  Maximum  likelihood  can  be 

used  to  estimate  the  parameters. 

The  likelihood  function  is 

I  =  nfLfa)exp{-A(r4  (Al) 

k= 1  /=1 

Taking  logarithms  results  in  the  log  likelihood 

K  K.  K  w  K 

£(r,S)  =  ln(y)YJ nk  +  In (S )  ~ 0 Z “ ^Z Tk  (A2) 

k= 1  k= I  A-=l  i=\  k= 1 


Partial  differentiation  of  the  log-likelihood  results  in 


dy 


~  Z  nk  -  Z  Tk 

r  k= i  k= i 


(A3) 


Setting  the  partial  derivative  equal  to  0  and  solving  results  in 

K 

Z 11  k 

r  =  Jf—  (A4) 

Z  if 

k= 1 
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The  second  derivatives  can  be  used  to  obtain  estimates  of  the  asymptotic  variance  of  estimates  of 
y  and  8  using  Fisher  information  evaluated  at  the  parameter  estimates.  (Crowder,  1991) 
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EXHIBIT  2  APPENDIX  D  VISUAL  BASIC  FOR  APPLICATIONS  CODING 


Sub  CBM() 


'  CBM  Macro 

'  Macro  recorded  3/31/2006  by  Stephen  E.  Gauthier 

l  . 

'  Documentation:  LT  Jason  Kratz  (NPS  student)  provided  exceptional  , 

'  assistance  with  this  code  by  proofreading  and  finding  many  ways  to 
'  improve  upon  it’s  processing  speed. 

'This  keeps  workbook  from  updating  every  iteration 
Application. ScreenUpdating  =  False 
With  Application 

.Calculation  =  xlManual 
.MaxChange  =  0.001 
End  With 

ActiveWorkbook.PrecisionAsDisplayed  =  False 

'This  computes  Theta  L  and  sets  number  of  missions  to  1250 
Sheets ( "Legacy  Process" ) .Select 
NumberMissions  =  1250 
NewMissionLength  =  0 

Worksheets ( "Legacy  Process" ). Range (Cells { 8 ,  1),  Cells(7  +  NumberMissions, 

1) ) .ClearContents 

Dim  Three JArray (9)  As  Double 

Dim  Four JArray ( 9 )  As  Double 

For  j  =  1  To  9 

Three JArray ( j )  =  Worksheets ( "Legacy  Process ") .Cells (3 ,  j) 

Four JArray ( j )  =  Worksheets ( "Legacy  Process" ) .Cells (4,  j) 

Next  j 

For)  i  =  1  To  NumberMissions 

NewMissionLength  =  NewMissionLength  +  4 

For  j  =  1  To  5 
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If  Three JArray ( j )  =  0  Then 

Worksheets ( "Legacy  Process" ) .Cells (7  +  i,  1)  =  Four JArray ( 1 ) 
End  If 


If  Three JArray ( j )  >  0  Then 
multiplier  =  0 
multiplier  = 

Application. WorksheetFunction.RoundDown(NewMissionLength  /  Three JArray ( j ) ,  0) 
Worksheets ( "Legacy  Process" > .Cells (1,  1)  =  multiplier 

End  If 

If  NewMissionLength  =  multiplier  *  Three JArray ( j )  Then 

Worksheets ( "Legacy  Process" ) .Cells (7  +  i,  1)  =  Four JArray ( j ) 

Elself  NewMissionLength  -  1  =  multiplier  *  Three JArray ( j )  Then 
Worksheets ( "Legacy  Process" ) .Cells (7  +  i,  1)  =  FourJArray ( j ) 

Elself  NewMissionLength  -  2  =  multiplier  *  Three JArray ( j )  Then 
.  Worksheets ( "Legacy  Process" ) .Cells (7  +  i,  1)  =  FourJArray (j ) 

Elself  NewMissionLength  -  3  =  multiplier  *  ThreeJArray ( j )  Then 
Worksheets ( "Legacy  Process" ) .Cells (7  +  i,  1)  =  FourJArray (j) 

End  If 

Next  j 

Next  i 

'This  clears  Data  Worksheet  below  Row  24  and  left  of  and  including  Column  36 

Sheets ( "Data" ) . Select 

M  =  NumberMissions 

N  =  36 

Worksheets ( "Data" ) .Range (Cells (25 ,  1),  Cells  (24  +  M,  N) ) .ClearContents 

'This  sets  the  number  of  iterations  to  perform 

Replications  =  Worksheets ( "RVs" ) .Cells (1,  2) 

For  v  =  1  To  Replications  < 

Calculate 


'This  clears  N,  M,  R,  and  AS  Columns  of  Legacy  Process  Worksheet 
Sheets ( "Legacy  Process”) .Select 
N  =  3 


Worksheets ( "Legacy  Process") . Range (Cells ( 8 , 
N) ) . ClearContents 

Worksheets ( "Legacy  Process") .Range (Cells (8 , 
N)  )  .ClearContents 

Worksheets ( "Legacy  Process") . Range (Cells ( 8 , 
N) ) .ClearContents 

Worksheets ( "Legacy  Process") .Range (Cells (8, 
N)  ) .ClearContents 


12) ,  Cells (7  +  M, 

24) ,  Cells (7  +  M, 

35) ,  Cells (7  +  M, 

46) ,  Cells (7  +  M, 


11  + 

23  + 

34  + 

45  + 


'This  computes  Sum  of  Dij  and  Rij  for  cases  <=  and  >  M 
Sheets ( "Legacy  Process") .Select 
NCount  =  0 
RTime  =  0 
N  =  7 

For  i  =  1  To  M 

If  Worksheets ( "Legacy  Process ") .Cells (7  +  i,  4)  >  0  Then 
NewNCount  =  0 
NCount  =  0 
NewNCount2  =  0 
NCount2  =  0 
NewNCount3  =  0 
NCount3  =  0 
NewNCount4  =  0 
NCount4  =  0 
NewRTime  =  0 
RTime  =  0 
NewRTime2  =  0 
RTime2  =  0 
NewASTime  =  0 
ASTime  =  0 
ASTime2  =  0 
NewASTime2  =  0 

temp2  =  Worksheets ( "Legacy  Process" ) .Cells (7  +  i,  2) 
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For  j  =  1  To  N 

temp4  =  Worksheets ( "Legacy  Process" ) .Cells (7  +  i, 
templ6  =  Worksheets ( "Legacy  Process" ) .Cells (7  +  i, 
temp27  =  Worksheets ( "Legacy  Process" ) .Cells (7  +  i, 
temp38  =  Worksheets ( "Legacy  Process" ) .Cells (7  +  i, 

If  temp4  <  tempi 6  Then 

If  temp4  >  0  And  temp2  +  temp4  <=  M  +  1  Then 
NewNCount  =  1 

NCount  =  NewNCount  +  NCount 
End  If 

If  temp4  >  0  And  temp2  +  temp4  >  M  +  1  Then 
NewNCount2  =  1 

NCount2  =  NewNCount2  +  NCount2 
End  If 

If  temp2  +  temp4  <=  NumberMissions  +  1  Then 
NewRTime  =  temp27 
RTime  =  NewRTime  +  RTime 
NewASTime  =  temp38 
ASTime  =  NewASTime  +  ASTime 
End  If 

If  temp2  +  temp4  >  NumberMissions  +  1  Then 
NewRTime2  =  temp27 
RTime2  =  NewRTime2  +  RTime2 
NewASTime2  =  temp38 
ASTime2  =  NewASTime2  +  ASTime2 
End  If 
Else 

If  templ6  >  0  And  temp2  +  templ6  <=  M  +  1  Then 
NewNCount3  =  1 

NCount3  =  NewNCount3  +  NCount3 
End  If 

If  tempi 6  >  0  And  temp2  +  tempi 6  >  M  +  1  Then 
NewNCount4  =  1 

NCount4  =  NewNCount4  +  NCount4 
End  If 

If  temp2  +  templ6  <=  NumberMissions  +  1  Then 


4  +  j) 

16  +  j) 
27  +  j) 
38  +  j) 
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NewRTime  =  temp27 
RTime  =  NewRTime  +  RTime 
NewASTime  =  temp38 
ASTime  =  NewASTime  +  ASTime 
End  If 


If  temp2  +  templ6  >  NumberMissions  +  1  Then 
NewRTime2  =  temp27 
RTime2  =  NewRTime2  +  RTime2 
NewASTime2  =  temp3  8 
ASTime2  =  NewASTime2  +  ASTime2 
End  If 
End  If 


Next  j 

Worksheets ( "Legacy 
Worksheets ( " Legacy 
Worksheets ( " Legacy 
Worksheets ( "Legacy 
Worksheets ( "Legacy 
Worksheets ( "Legacy 
Worksheets ( "Legacy 
Worksheets ( "Legacy 


Process11)  .Cells  (7 
Process") .Cells (7 
Process" ) .Cells (7 
Process") .Cells (7 
Process") .Cells (7 
Process" ) .Cells  (7 
Process" ) .Cells  (7 
Process" ) .Cells  (7 


+ 

i, 

12) 

= 

NCount 

+ 

i, 

13) 

= 

NCount2 

+ 

i, 

24) 

= 

NCount 3 

+ 

i, 

25) 

= 

NCount4 

+ 

i, 

35) 

= 

RTime 

+ 

i, 

36) 

= 

RTime2 

+ 

i, 

46) 

= 

ASTime 

+ 

i, 

47) 

= 

AS Time 2 

End  If 


Worksheets ( "Legacy  Process" ) .Cells (7  +  i, 
Process" ) .Cells (7  +  i,  12)  +  Worksheets ( "Legacy 
Worksheets ( "Legacy  Process" ) .Cells (7  +  i, 
Process" ) .Cells  (7  +  i,  24)  +  Worksheets ( "Legacy 
Worksheets ( "Legacy  Process" ) .Cells (7  +  i, 
Process" ) .Cells  (7  +  i,  35)  +  Worksheets ( "Legacy 
Worksheets ( "Legacy  Process" ) .Cells {7  +  i, 
Process" ) .Cells  (7  +  i,  46)  +  Worksheets ( "Legacy 
Next  i 


14)  =  Worksheets ( "Legacy 
Process ") .Cells  (7  +  i,  13) 
26)  =  Worksheets ( "Legacy 
Process ") .Cells (7  +  i,  25) 
37)  =  Worksheets ( "Legacy 
Process "). Cells (7  +  i,  36) 
48)  =  Worksheets ( "Legacy 
Process" ) .Cells (7  +  i,  47) 


'This  determines  if  there  was  an  immediate  Mission  Abort 
Sheets ( "RVs" ) .Select 
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For  i  =  1  To  M 

If  Application. WorksheetFunction . Countlf (Range(Cells (4  +  i,  6),  Cells(4 
+  i,  12) ) ,  "=  1”)  >=  1  Then 

Sheets ( "Legacy  Process") .Select 

Worksheets ( "Legacy  Process" ) .Cells (7  +  i,  16)  =  1 

Sheets { " RVs " ) .Select 

Else 

Worksheets ( "Legacy  Process" ) .Cells (7  +  i,  16)  =  0 
End  If 
Next  i 

' This  produces  the  summed  outputs  for  the  Legacy  Spreadsheet 

Sheets ( "Legacy  Process") .Select 

Worksheets ( "Legacy  Process" ) .Cells (1,  15)  = 

Application. WorksheetFunction. Sum(Range {Cells (8 ,  12),  Cells(7  +  M,  12))) 
Worksheets ( "Legacy  Process" ) .Cells (2,  15)  = 

Application. WorksheetFunction. Sum(Range (Cells {8,  13),  Cells (7  +  M,  13))) 
Worksheets ( "Legacy  Process "). Cells (3 ,  15)  = 

Application. WorksheetFunction. Sum (Range (Cells (8,  14),  Cells (7  +  M,  14))) 
Worksheets ( "Legacy  Process" ) .Cells (4,  15)  = 

Application. WorksheetFunction. Sum (Range (Cells (8,  24),  Cells  (7  +  M,  24))) 
Worksheets ( "Legacy  Process" ) .Cells (5,  15)  = 

Application. WorksheetFunction. Sum (Range (Cells (8 ,  25),  Cells  (7  +  M,  25))) 
Worksheets ( "Legacy  Process" ) .Cells (6,  15)  = 

Application. WorksheetFunction. Sum (Range (Cells (8,  26),  Cells (7  +  M,  26))) 
Worksheets ( "Legacy  Process" ) .Cells (1,  18)  = 

Application. WorksheetFunction. Sum (Range (Cells (8,  35),  Cells (7  +  M,  35))) 
Worksheets ( "Legacy  Process "). Cells (2 ,  18)  = 

Application. WorksheetFunction. Sum (Range (Cells (8,  36),  Cells (7  +  M,  36))) 
Worksheets ( "Legacy  Process" ) .Cells (3 ,  18)  = 

Application. WorksheetFunction. Sum(Range (Cells (8,  37),  Cells (7  +  M,  37))) 
Worksheets ( "Legacy  Process" ) .Cells (4 ,  18)  = 

Application. WorksheetFunction. Sum(Range (Cells (8 ,  46),  Cells (7  +  M,  46))) 
Worksheets ( "Legacy  Process" ) .Cells (5,  18)  = 

Application. WorksheetFunction. Sum (Range (Cells (8,  47),  Cells  (7  +  M,  47))) 
Worksheets ( "Legacy  Process" ) .Cells (6,  18)  = 

Application. WorksheetFunction. Sum(Range (Cells (8,  48),  Cells  (7  +  M,  48))) 
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'This  clears  Summed  D,  M,  R,  and  AS  Columns  of  CBM+  Process  Worksheet 
Sheets ( "CBM+  Process "). Select 
N  =  3 


Worksheets ( "CBM+  Process") .Range (Cells (8, 
N) ) .ClearContents 

Worksheets ( "CBM+  Process") .Range (Cells {8, 
N) ) .ClearContents 

Worksheets ( "CBM+  Process") .Range (Cells (8, 
N) ) .ClearContents 

Worksheets ( "CBM+  Process") .Range (Cells (8, 
N) ) .ClearContents 


12) ,  Cells (7  +  M,  11  + 

24) ,  Cells  (7  +  M,  23  + 

35) ,  Cells (7  +  M,  34  + 

46) ,  Cells (7  +  M,  45  + 


'This  computes  Sum  of  Dij  and  Sum  of  Rij  for  cases  <=  and  >  M 
Sheets("CBM+  Process '').  Select 
NCount  =  0 
M  =  NumberMissions 
N  =  7 

For  i  =  1  To  M 

If  Worksheets ( "CBM+  Process” ) .Cells (7  +  i,  4)  >  0  Then 
NewNCount  =  0 
NCount  =  0 
NewNCount2  =  0 
NCount2  =  0 
NewNCount 3  =  0 
NCount3  =  0 
NewNCount4  =  0 
NCount4  =  0 
NewRTime  =  0 
RTime  =  0 
NewRTime2  =  0 
RTime2  =  0 
NewASTime  =  0 
ASTime  =  0 
ASTime2  =  0 
NewASTime2  =  0 


96 


temp2  =  Worksheets ( "CBM+  Process "). Cells (7  +  i,  2) 

For  j  =  1  To  N 

temp4  =  Worksheets ( "CBM +  Process ") .Cells  (7  +  i,  4  +  j) 
templ6  =  Worksheets ( "CBM+  Process ”). Cells (7  +  i,  16  +  j) 

temp27  =  Worksheets  ( 11 CBM+  Process" )  .Cells  (7  +  i,  27  +  j) 

temp38  =  Worksheets ( "CBM+  Process" ) .Cells (7  +  i,  38  +  j) 

If  temp4  <  tempi 6  Then 

If  temp4  >  0  And  temp2  +  temp4  <=  M  +  1  Then 
NewNCount  =  1 

NCount  =  NewNCount  +  NCount 
End  If 

If  temp4  >  0  And  temp2  +  temp4  >  M  +  1  Then 
NewNCount2  =  1 

NCount2  =  NewNCount2  +  NCount2 
End  If 

If  temp2  +  temp4  <=  NumberMissions  +  1  Then 
NewRTime  =  temp27 
RTime  =  NewRTime  +  RTime 
NewASTime  =  temp38 
ASTime  =  NewASTime  +  ASTime 
End  If 

If  temp2  +  temp4  >  NumberMissions  +  1  Then 
NewRTime2  =  temp27 
RTime 2  =  NewRTime 2  +  RTime 2 
NewASTime2  =  temp38 
ASTime2  =  NewASTime2  +  ASTime2 
End  If 
Else 

If  templ6  >  0  And  temp2  +  templ6  <=  M  +  1  Then 
NewNCount3  =  1 

NCount3  =  NewNCount3  +  NCount3 
End  If 

If  tempi 6  >  0  And  temp2  +  tempi 6  >  M  +  1  Then 
NewNCount4  =  1 

NCount4  =  NewNCount4  +  NCount4 
End  If 
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If  temp2  +  templ6  <=  NumberMissions  +  1  Then 
NewRTime  =  temp27 
RTime  =  NewRTime  +  RTime 
NewASTime  =  temp3  8 
ASTime  =  NewASTime  +  ASTime 


End  If 


If  temp2  +  templ6  >  NumberMissions  +  1  Then 
NewRTime2  =  temp27 
RTime2  =  NewRTime2  +  RTime2 
NewASTime2  =  temp38 
ASTime2  =  NewASTime2  +  ASTime2 
End  If 
End  If 

Next  j 


Worksheets ( "CBM+ 

Process “ ) 

.Cells  (7 

+ 

i, 

12) 

= 

NCount 

Worksheets ( "CBM+ 

Process " ) 

.Cells  (7 

+ 

i, 

13) 

= 

NCount2 

Worksheets ( " CBM+ 

Process" ) 

.Cells  (7 

+ 

i  i 

24) 

= 

NCount 3 

Worksheets ( "CBM+ 

Process" ) 

.Cells  (7 

+ 

i, 

25) 

= 

NCount4 

Worksheets ( "CBM+ 

Process" ) 

.Cells  (7 

+ 

i, 

35) 

= 

RTime 

Worksheets ( "CBM+ 

Process" ) 

.Cells  (7 

+ 

i, 

36) 

= 

RTime2 

Worksheets ( " CBM+ 

Process" ) 

.Cells  (7 

+ 

i, 

46) 

= 

ASTime 

Worksheets ( "CBM+ 

Process " ) 

.Cells  {7 

+ 

i, 

47) 

= 

AS Time 2 

End  If 

Worksheets ( "CBM+ 
Process" ) .Cells  (7  +  i, 
Worksheets ( "CBM+ 
Process" ) .Cells (7  +  i. 
Worksheets ( "CBM+ 
Process" ) .Cells  (7  +  i. 
Worksheets { "CBM+ 
Process” ) .Cells (7  +  i, 
Worksheets ( "CBM+ 
Process" ) .Cells (7  +  i. 


Process" ) .Cells (7  +  i, 
12)  +  Worksheets ( "CBM+ 
Process" ) .Cells (7  +  i, 
24)  +  Worksheets ( "CBM+ 
Process" ) .Cells (7  +  i, 
35)  +  Worksheets ( "CBM+ 
Process" ) .Cells (7  +  i, 
46)  +  Worksheets ( "CBM+ 
Process" ) .Cells (7  +  i, 
37)  +  Worksheets { "CBM+ 


14)  =  Worksheets ( "CBM+ 
Process ") .Cells (7  +  i,  13) 
26)  =  Worksheets ( "CBM+ 
Process" ) .Cells (7  +  i,  25) 
37)  =  Worksheets ( "CBM+ 
Process" ) .Cells (7  +  i,  36) 
48)  =  Worksheets ( "CBM+ 
Process" ) .Cells (7  +  i,  47) 
52)  =  Worksheets ( "CBM+ 
Process" ) .Cells (7  +  i,  51) 


Next  i 
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'This  determines  if  there  was  an  immediate  Mission  Abort 
Sheets ( " RVs " ) .Select 

For  i  =  1  To  M 

If  Application. WorksheetFunction.Countlf (Range (Cells (4  +  i,  6),  Cells(4 
+  i,  12) )  ,  ■=  1" )  >=  1  Then 

Sheets ( "CBM+  Process "). Select 

Worksheets ( "CBM+  Process" ) .Cells (7  +  i,  16)  =  1 

Sheets ("RVs") .Select 

Else 

Worksheets ( "CBM+  Process" ) .Cells (7  +  i,  16)  =  0 
End  If 
Next  i 

' This  produces  the  summed  outputs  for  the  CBM+  Spreadsheet 
Sheets ("CBM+  Process "). Select 


Worksheets ( "CBM+  Process" ) .Cells (1,  15)  = 
Application.WorksheetFunction. Sum (Range (Cells (8 , 
Worksheets ( "CBM+  Process" ) .Cells (2 ,  15)  = 
Application.WorksheetFunction . Siam (Range (Cells ( 8 , 
Worksheets ( “CBM+  Process" ) .Cells (3 ,  15)  = 
Application . WorksheetFunction . Sum (Range (Cells ( 8 , 
Worksheets ( "CBM+  Process" ) .Cells (4,  15)  = 
Application.WorksheetFunction . Sum (Range (Cells ( 8 , 
Worksheets ( "CBM+  Process" ) .Cells (5,  15)  = 
Application. WorksheetFunction. Sum (Range (Cells (8, 
Worksheets ( "CBM+  Process" ) .Cells (6,  15)  = 
Application.WorksheetFunction. Sum (Range (Cells (8 , 
Worksheets ( "CBM+  Process" ) .Cells (1,  18)  = 
Application.WorksheetFunction . Sum (Range (Cells ( 8 , 
Worksheets ( "CBM+  Process ") .Cells  (2 ,  18)  = 
Application.WorksheetFunction. Sum (Range (Cells (8, 
Worksheets ( "CBM+  Process "). Cells (3 ,  18)  = 
Application.WorksheetFunction . Sum (Range (Cells ( 8 , 
Worksheets ( "CBM+  Process" ) .Cells (4,  18)  = 
Application.WorksheetFunction. Sum (Range (Cells (8, 


12)  ,  Cells (7  +  M,  12) ) ) 

13)  ,  Cells (7  +  M,  13) ) ) 

14)  ,  Cells  (7  +  M,  14) ) ) 

24)  ,  Cells  (7  +  M,  24) ) ) 

25)  ,  Cells (7  +  M,  25) ) ) 

26)  ,  Cells (7  +  M,  26) ) ) 

35)  ,  Cells (7  +  M,  35) ) ) 

36)  ,  Cells (7  +  M,  36) ) ) 

37)  ,  Cells (7  +  M,  37) ) ) 

46) ,  Cells (7  +  M,  46) ) ) 
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Worksheets ( "CBM+  Process" ) .Cells (5,  18)  = 
Application. WorksheetFunction. Sum (Range (Cells (8, 
Worksheets ( "CBM+  Process" ) .Cells (6,  18)  = 
Application. WorksheetFunction. Sum (Range (Cells (8, 
Worksheets  ( "CBM+  Process1' )  .Cells  (1,  21)  = 
Application. WorksheetFunction. Sum  (Range (Cells (8 , 
Worksheets ( "CBM+  Process" ) .Cells (2,  21)  = 

Appl ication. WorksheetFunction. Sum  (Range (Cells (8 , 
Worksheets ( "CBM+  Process" ) .Cells (3 ,  21)  = 
Application. WorksheetFunction. Sum(Range (Cells (8, 


47) ,  Cells  (7 

48) ,  Cells  (7 

50) ,  Cells  (7 

51) ,  Cells  (7 

52) ,  Cells  (7 


+  M,  47))) 

+  M,  48))) 

+  M,  50))) 

+  M,  51))) 

+  M,  52)  )  ) 


'This  transposes  data  onto  Data  Worksheet 
Sheets ( "Legacy  Process") .Select 
Range (Cells (1 ,  15),  Cells (6,  15)). Select 
Selection . Copy 
Sheets ( "Data" ) .Select 

Range (Cells (24  +  v,  1),  Cells (24  +  v,  6)). Select 

Selection. PasteSpecial  Paste : =xlPasteAll,  Operation: =xlNone,  SkipBlanks : 
False,  Transpose : =True 


Sheets ( "Legacy  Process") .Select 
Range (Cells (1,  18),  Cells (6,  18)). Select 
Selection . Copy 
Sheets ( "Data" ) . Select 

Range (Cells  (24  +  v,  7),  Cells (24  +  v,  12)). Select 

Selection. PasteSpecial  Paste :=xlPasteAll,  Operation: =xlNone,  SkipBlanks: 
False,  Transpose :=True 

'This  transposes  data  onto  Data  Worksheet 
Sheets("CBM+  Process" ). Select 
Range (Cells (1,  15),  Cells (6,  15)). Select 
Selection . Copy 
Sheets ( "Data" ) .Select 

Range (Cells (24  +  v,  22),  Cells(24  +  v,  27)). Select 

Selection . PasteSpecial  Paste : =xlPasteAll ,  Operation : =xlNone ,  SkipBlanks: 
False,  Transpose : =True 

Sheets("CBM+  Process "). Select 
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Range (Cells  (1,  18),  Cells(6,  18)). Select 
Selection.  Copy- 
Sheets  ( "Data" ) . Select 

Range (Cells (24  +  v,  28),  Cells (24  +  v,  33)). Select 

Selection. PasteSpecial  Paste :=xlPasteAll,  Operation :=xlNone,  SkipBlanks : =  _ 
False,  Transpose : =True 

Sheets("CBM+  Process "). Select 
Range (Cells  (1 ,  21),  Cells(3,  21)). Select 
Selection . Copy 
Sheets ( "Data" ) .Select 

Range (Cells (24  +  v,  34),  Cells(24  +  v,  36)). Select 

Selection. PasteSpecial  Paste : =xlPasteAll ,  Operation: =xlNone,  SkipBlanks:=  _ 
False,  Transpose : =True 


Next  v 

'This  determines  means  and  CIs 
Sheets ( "Data" ) .Select 
Worksheets ( "Data" ) .Cells (2 ,  3)  = 

Application. WorksheetFunction. Average (Range (Cells (25 ,  1),  Cells(24  +  v,  1))) 
Worksheets ( "Data" ) .Cells (3 ,  3)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  2),  Cells  (24  +  v,  2))) 
Worksheets ( "Data" ) .Cells (4,  3)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  3),  Cells  (24  +  v,  3))) 
Worksheets ( "Data" ) .Cells (5,  3)  = 

Application. WorksheetFunction. Average (Range (Cells (25 ,  4),  Cells(24  +  v,  4))) 
Worksheets ( "Data" ) .Cells  (6 ,  3)  = 

Application. WorksheetFunction. Average (Range (Cells (25 ,  5),  Cells(24  +  v,  5))) 
Worksheets ( "Data" ) .Cells (7 ,  3)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  6),  Cells  (24  +  v,  6))) 
Worksheets ( "Data" ) .Cells (8 ,  3)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  7),  Cells  (24  +  v,  7))) 
Worksheets ( "Data" ) .Cells  (9 ,  3)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  8),  Cells (24  +  v,  8))) 
Worksheets ( "Data" ) .Cells (10 ,  3)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  9),  Cells (24  +  v,  9))) 
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Worksheets ( "Data" ) .Cells (11 ,  3)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  10),  Cells(24  +  v, 
10))) 

Worksheets ( "Data” ) .Cells (12 ,  3)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  11),  Cells  (24  +  v, 
11)  )  ) 

Worksheets ( "Data" ) .Cells (13 ,  3)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  12),  Cells(24  +  v, 
12)  ) ) 

Worksheets ( "Data" ) .Cells (2 ,  11)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  22),  Cells  (24  +  v, 

22)  )  ) 

Worksheets ( "Data" ) .Cells (3 ,  11)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  23),  Cells(24  +  v, 

23) )) 

Worksheets ( "Data" ) .Cells (4 ,  11)  = 

Application. WorksheetFunction. Average (Range (Cells (25 ,  24),  Cells(24  +  v, 

24) )) 

Worksheets ( "Data" ) .Cells (5,  11)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  25),  Cells(24  +  v, 

25) )) 

Worksheets ( "Data" ) .Cells  (6,  11)  = 

Application. WorksheetFunction. Average (Range (Cells (25 ,  26),  Cells(24  +  v, 

26) )) 

Worksheets ( "Data" ) .Cells (7 ,  11)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  27),  Cells(24  +  v, 

27) )) 

Worksheets ( "Data" ) .Cells (8,  11)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  28),  Cells(24  +  v, 

28)  )  ) 

Worksheets ( "Data" ) .Cells (9 ,  11)  = 

Application. WorksheetFunction. Average (Range (Cells (25 ,  29),  Cells(24  +  v, 

29)  )  ) 

Worksheets ( "Data" ) .Cells  (10 ,  11)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  30),  Cells  (24  +  v, 

30)  )  ) 
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Worksheets ( "Data" ) .Cells (11,  11)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  31),  Cells(24  +  v, 

31) )) 

Worksheets ( "Data" ) .Cells (12 ,  11)  = 

Application. WorksheetFunction. Average (Range (Cells (25 ,  32),  Cells(24  +  v, 

32)  )  ) 

Worksheets ( "Data" ) .Cells (13 ,  11)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  33),  Cells (24  +  v, 

33)  )  ) 

Worksheets ( "Data" ) .Cells (14 ,  11)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  34),  Cells(24  +  v, 

34) )) 

Worksheets ( "Data" ) .Cells (15 ,  11)  = 

Application. WorksheetFunction. Average (Range (Cells (25 ,  35),  Cells(24  +  v, 

35) )) 

Worksheets ( "Data" ) .Cells (16 ,  11)  = 

Application. WorksheetFunction. Average (Range (Cells (25,  36),  Cells(24  +  v, 

36)  )  ) 


If  Worksheets ( "Data" ) .Cells (2 ,  3)  >  0  Then 
Worksheets ( "Data" ) .Cells (2 ,  5)  = 

Application. WorksheetFunction . StDev(Range (Cells (25 ,  1),  Cells  (24  +  v,  1))) 
Else:  Worksheets ( "Data" ) .Cells (2 ,  5)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (3 ,  3)  >  0  Then 
Worksheets ( "Data" ) .Cells (3 ,  5)  = 

Application. WorksheetFunction. StDev(Range (Cells (25,  2),  Cells  (24  +  v,  2))) 
Else:  Worksheets ( "Data" ) .Cells (3 ,  5)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (4 ,  3)  >  0  Then 
Worksheets ( "Data" ) .Cells (4,  5)  = 

Application. WorksheetFunction. StDevfRange (Cells (25,  3),  Cells(24  +  v,  3))) 
Else:  Worksheets ( "Data" ) .Cells (4 ,  5)  =  0 

End  If 
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If  Worksheets ( "Data" ) .Cells (5,  3)  >  0  Then 
Worksheets ( "Data" ) .Cells (5 ,  5)  = 

Application. WorksheetFunction. StDev{Range (Cells (25,  4),  Cells(24  +  v,  4))) 
Else:  Worksheets ( "Data” ) .Cells (5 ,  5)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (6,  3)  >  0  Then 
Worksheets ( "Data" ) .Cells (6,  5)  = 

Application. WorksheetFunction. StDev(Range (Cells (25,  5),  Cells  (24  +  v,  5))) 
Else:  Worksheets ( "Data" ) .Cells (6,  5)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (7 ,  3)  >  0  Then 
Worksheets ( "Data" ) .Cells  (7 ,  5)  = 

Application . WorksheetFunction. StDev (Range (Cells (25 ,  6),  Cells (24  +  v,  6))) 
Else:  Worksheets ( "Data" ) .Cells (7 ,  5)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (8 ,  3)  >  0  Then 
Worksheets ( "Data" ) .Cells (8 ,  5)  = 

Application. WorksheetFunction. StDev (Range (Cells (25,  7),  Cells  (24  +  v,  7))) 
Else:  Worksheets ( "Data" ) .Cells (8 ,  5)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (9 ,  3)  >  0  Then 
Worksheets ( "Data" ) .Cells (9 ,  5)  = 

Application.WorksheetFunction. StDev (Range (Cells (25,  8),  Cells  (24  +  v,  8))) 
Else:  Worksheets ( "Data" ) .Cells (9,  5)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (10 ,  3)  >  0  Then 
Worksheets ( "Data" ) .Cells (10 ,  5)  = 

Application. WorksheetFunction. StDev(Range (Cells (25,  9),  Cells (24  +  v,  9))) 
Else:  Worksheets ( "Data" ) .Cells (10,  5)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (11 ,  3)  >  0  Then 
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Worksheets { "Data" ) .Cells (11,  5)  = 

Application. WorksheetFunction.StDev (Range (Cells (25,  10),  Cells  (24  +  v,  10))) 
Else:  Worksheets ( "Data" ) .Cells (11 ,  5)  =  0 

End  If 

If  Worksheets ( “Data" ). Cells (12 ,  3)  >  0  Then 
Worksheets ( "Data" ) .Cells (12 ,  5)  = 

Application. WorksheetFunction.StDev (Range (Cells (25,  11),  Cells  (24  +  v,  11))) 
Else:  Worksheets ( "Data" ) .Cells (12 ,  5)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (13 ,  3)  >  0  Then 
Worksheets ( "Data" ) .Cells  (13 ,  5)  = 

Application. WorksheetFunction . StDev (Range (Cells (25,  12),  Cells (24  +  v,  12))) 
Else:  Worksheets ( "Data" ) .Cells (13 ,  5)  =  0 

End  If 


If  Worksheets ( "Data" ) .Cells (2 ,  11)  >  0  Then 
Worksheets ( "Data" ) .Cells (2 ,  13)  = 

Application. WorksheetFunction. StDev (Range (Cells (25,  22),  Cells (24  +  v,  22))) 
Else:  Worksheets ( "Data" ) .Cells (2,  13)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (3 ,  11)  >  0  Then 
Worksheets ( "Data" ) .Cells (3 ,  13)  = 

Application. WorksheetFunction. StDev (Range (Cells (25,  23),  Cells  (24  +  v,  23))) 
Else:  Worksheets ( "Data" ) .Cells (3 ,  13)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (4,  11)  >  0  Then 
Worksheets ( "Data" ) .Cells (4 ,  13)  = 

Application. WorksheetFunction. StDev (Range (Cells (25,  24),  Cells (24  +  v,  24))) 
Else:  Worksheets ( "Data" ) .Cells (4 ,  13)  =0 

End  If 

If  Worksheets ( "Data" ) .Cells (5,  11)  >  0  Then 
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Worksheets ( “Data" ) .Cells (5 ,  13)  = 

Application. WorksheetFunction.StDev(Range (Cells (25,  25),  Cells (24  +  v,  25))) 
Else:  Worksheets ( "Data" ) .Cells (5,  13)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (6,  11)  >  0  Then 
Worksheets ( "Data" ) .Cells (6,  13)  = 

Application. WorksheetPunction. StDev (Range (Cells (25,  26),  Cells  (24  +  v,  26))) 
Else:  Worksheets ( "Data" ) .Cells ( 6 ,  13)  =  0 

End  If 

If  Worksheets ( "Data” ) .Cells (7 ,  11)  >  0  Then 
Worksheets ( "Data" ) .Cells (7 ,  13)  = 

Application. WorksheetFunction. StDev (Range (Cells (25,  27),  Cells (24  +  v,  27))) 
Else:  Worksheets ( "Data" ) .Cells (7 ,  13)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (8 ,  11)  >  0  Then 
Worksheets ( "Data" ) .Cells  (8 ,  13)  = 

Application. WorksheetFunction. StDev (Range (Cells (25,  28),  Cells  (24  +  v,  28))) 
Else:  Worksheets ( "Data" ) .Cells (8,  13)  =  0 

End  If 

If  Worksheets ( "Data" ). Cells (9 ,  11)  >  0  Then 
Worksheets ( "Data" ) .Cells (9 ,  13)  = 

Application. WorksheetFunction. StDev(Range (Cells (25,  29),  Cells (24  +  v,  29))) 
Else:  Worksheets ( "Data" ) .Cells (9 ,  13)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (10 ,  11)  >  0  Then 
Worksheets ( "Data" ) .Cells (10 ,  13)  = 

Application. WorksheetFunction. StDev (Range (Cells (25,  30),  Cells  (24  +  v,  30))) 
Else:  Worksheets ( "Data" ) .Cells (10 ,  13)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (11,  11)  >  0  Then 
Worksheets ( "Data" ) .Cells (11,  13)  = 

Application. WorksheetFunction. StDev (Range (Cells (25,  31),  Cells (24  +  v,  31))) 
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Else:  Worksheets ( "Data" ) .Cells (11,  13)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (12 ,  11)  >  0  Then 
Worksheets ( "Data" ) .Cells (12 ,  13)  = 

Application. WorksheetFunction. StDev (Range (Cells (25,  32),  Cells  (24  +  v,  32))) 
Else:  Worksheets ( "Data" ) .Cells (12 ,  13)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (13 ,  11)  >  0  Then 
Worksheets ( "Data" ) .Cells (13 ,  13)  = 

Application. WorksheetFunction. StDev (Range (Cells (25,  33),  Cells  (24  +  v,  33))) 
Else:  Worksheets ( "Data" ) .Cells (13  ,  13)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (14,  11)  >  0  Then 
Worksheets ( "Data" ) .Cells (14,  13)  = 

Application. WorksheetFunction. StDev (Range (Cells (25,  34),  Cells (24  +  v,  34))) 
Else:  Worksheets ( "Data" ) .Cells (14,  13)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (15,  11)  >  0  Then 
Worksheets ( "Data" ) .Cells (15,  13)  = 

Application. WorksheetFunction. StDev (Range (Cells (25,  35),  Cells(24  +  v,  35))) 
Else:  Worksheets ( "Data" ) .Cells (15,  13)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (16,  11)  >  0  Then 
Worksheets ( "Data" ) .Cells  (16,  13)  = 

Application .WorksheetFunction. StDev (Range (Cells (25 ,  36),  Cells  (24  +  v,  36))) 
Else:  Worksheets ( "Data" ) .Cells (16 ,  13)  =  0 

End  If 

If  Worksheets  ( "Data" )  .Cells  (2 ,'  5)  >  0  Then 
Worksheets ( "Data" ) .Cells  (2 ,  7)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells(2,  3)  - 
Application .WorksheetFunction. Confidence (0 . 05 ,  Worksheets ( "Data" ) . Cells (2 , 
5),  (v  -  1)),  0) 
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Worksheets ( "Data" ) .Cells (2 ,  8)  =  Worksheets ( "Data" ). Cells (2 ,  3)  + 
Application.WorksheetFunct ion. Confidence (0.05,  Worksheets ( "Data" ) .Cells (2 , 
5)  ,  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (2 ,  7)  =  0 
Worksheets ( "Data" ) .Cells (2 ,  8)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (3 ,  5)  >  0  Then 
Worksheets ( "Data" ) .Cells (3 ,  7)  = 

Appl ication.WorksheetFunction. Max (Worksheets ( "Data" ) .Cells(3,  3)  - 
Application. WorksheetFunction.Conf idence (0.05,  Worksheets ( "Data" ) . Cells (3 , 
5)  ,  (v  -  1)  )  ,  0) 

Worksheets ( "Data" ) .Cells (3 ,  8)  =  Worksheets ( "Data" ) .Cells (3 ,  3)  + 
Application. WorksheetFunction.Conf idence (0.05,  Worksheets ( "Data" ) . Cells (3 , 
5)  ,  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (3 ,  7)  =  0 
Worksheets ( "Data" ) .Cells (3 ,  8)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (4,  5)  >  0  Then 
Worksheets ( "Data" ) .Cells (4 ,  7)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells(4,  3)  - 
Application. WorksheetFunction.Conf idence (0 . 05,  Worksheets ( "Data" ) .Cells (4, 
5)  ,  (v  -  1)  )  ,  0) 

Worksheets ( "Data" ) .Cells (4,  8)  =  Worksheets ( "Data" ) .Cells (4 ,  3)  + 
Application .WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) . Cells  (4 , 
5)  ,  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (4 ,  7)  =  0 
Worksheets ( "Data" ) .Cells (4,  8)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (5 ,  5)  >  0  Then 
Worksheets ( "Data" ) .Cells (5,  7)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells (5,  3)  - 
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Application. WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) .Cells (5, 
5),  (v  -  1)),  0) 

Worksheets ( "Data" ) .Cells (5,  8)  =  Worksheets ( "Data" ) .Cells (5,  3)  + 
Application. WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) .Cells (5, 
5)  ,  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (5,  7)  =  0 
Worksheets ( "Data" ) .Cells (5,  8)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (6 ,  5)  >  0  Then 
Worksheets ( "Data" ) .Cells (6 ,  7)  = 

Application .WorksheetFunction .Max (Worksheets ( "Data" ) .Cells (6,  3)  - 
Application. WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) .Cells (6, 
5),  (v  -  1)),  0) 

Worksheets ( "Data" ) .Cells (6,  8)  =  Worksheets ( "Data" ) .Cells (6,  3)  + 

Appl ication. WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) . Cells ( 6 , 
5)  ,  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (6,  7)  =  0 
Worksheets ( "Data" ) .Cells (6,  8)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (7 ,  5)  >  0  Then 
Worksheets ( "Data" ) .Cells (7 ,  7)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells(7,  3)  - 
Application. Wo rksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) . Cells (7 , 
5)  ,  (v  -  1)  )  ,  0) 

Worksheets ( "Data" ) .Cells (7 ,  8)  =  Worksheets ( "Data" ) .Cells (7 ,  3)  + 
Application. WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) .Cells (7 , 
5),  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (7 ,  7)  =  0 
Worksheets ( "Data" ) .Cells (7 ,  8)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (8 ,  5)  >  0  Then 
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Worksheets { "Data" ) .Cells (8 ,  7)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells(8,  3)  - 
Application. Wo rksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) . Cells ( 8 , 
5),  (v  -  D),  0) 

Worksheets ( "Data" ) .Cells (8,  8)  =  Worksheets ( "Data" ) .Cells ( 8 ,  3)  + 
Application .WorksheetFunction .Confidence (0.05,  Worksheets ( "Data" ) . Cells ( 8 , 
5),  v  -  1) 

Else 

Worksheets { "Data" ) .Cells (8 ,  7)  =  0 
Worksheets ( "Data" ) .Cells (8 ,  8)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (9,  5)  >  0  Then 
Worksheets ( "Data" ) .Cells (9,  7)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells (9,  3)  - 

Application. WorksheetFunction. Confidence(0 . 05,  Worksheets ( "Data" ) .Cells (9, 
5),  (v  -  1)),  0) 

Worksheets ( "Data" ) .Cells (9,  8)  =  Worksheets ( "Data" ) .Cells (9 ,  3)  + 
Applicat ion. WorksheetFunction. Conf idence (0 . 05 ,  Worksheets ( "Data" ) .Cells (9 , 
5),  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (9 ,  7)  =  0 
Worksheets ( "Data" ) .Cells (9,  8)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (10 ,  5)  >  0  Then 
Worksheets ( "Data" ) .Cells (10 ,  7}  = 

Application. WorksheetFunction .Max (Worksheets ( "Data" ) .Cells(10,  3)  - 
Applicat ion. WorksheetFunction .Conf idence (0.05,  Worksheets ( "Data" ) .Cells (10 , 
5)  ,  (v  -  1)  )  ,  0) 

Worksheets ( "Data" ) .Cells (10,  8)  =  Worksheets ( "Data" ) .Cells ( 10 ,  3)  + 
Applicat ion. Wo rksheetFunction. Conf idence (0.05,  Worksheets ( "Data" ) .Cells (10 , 
5)  ,  v  -  1) 

Else 

Worksheets  ( "Data11 )  .Cells  (10,  7)  =  0 
Worksheets ( "Data" ) .Cells (10 ,  8)  =  0 

End  If 
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If  Worksheets ( "Data" ) .Cells (11 ,  5)  >  0  Then 
Worksheets ( "Data" ) .Cells (11,  7)  = 

Application. WorksheetFunction. Max (Worksheets ("Data") .Cells (11,  3)  - 
Application.WorksheetFunction.Conf idence (0 .05,  Worksheets ( "Data" ) .Cells (11 , 
5),  (v  -  1)),  0) 

Worksheets ( "Data" ) .Cells (11,  8)  =  Worksheets ( "Data" ) .Cells (11,  3)  + 

Appl ication. WorksheetFunction. Conf idence (0 .05,  Worksheets ( "Data" ) .Cells (11 , 
5)  ,  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (11 ,  7)  =  0 
Worksheets ( "Data" ) .Cells (11,  8)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (12 ,  5)  >  0  Then 
Worksheets ( "Data" ) .Cells (12,  7)  = 

Application .WorksheetFunction .Max (Worksheets ( "Data" ) .Cells(12,  3)  - 
Application. WorksheetFunction. Confidence (0 . 05,  Worksheets ( "Data" ) .Cells (12 , 
5)  ,  (v  -  1)),  0) 

Worksheets ( "Data" ) .Cells (12 ,  8)  =  Worksheets ( "Data" ) .Cells (12,  3)  + 
Application. WorksheetFunction. Confidence (0 . 05,  Worksheets ( "Data" ) .Cells (12 , 
5),  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (12 ,  7}  =  0 
Worksheets ( "Data" ) .Cells (12 ,  8)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (13 ,  5)  >  0  Then 
Worksheets ( "Data" ) .Cells (13 ,  7)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells(13,  3)  - 
Application. WorksheetFunction. Conf idence (0.05,  Worksheets ( "Data" ) .Cells (13 , 
5)  ,  (v  -  1)),  0) 

Worksheets ( "Data" ) .Cells (13 ,  8)  =  Worksheets ( "Data" ) .Cells (13 ,  3)  + 
Application .WorksheetFunction. Conf idence (0.05,  Worksheets ( "Data" ) . Cells (13 , 
5)  ,  v  -  1) 

1  Else 

Worksheets ( "Data" ) .Cells (13 ,  7)  =  0 
Worksheets ( "Data" ) .Cells (13 ,  8)  =  0 
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End  If 


If  Worksheets ( "Data" ) .Cells (2 ,  13)  >  0  Then 
Worksheets ( "Data" ) .Cells (2 ,  15)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells(2,  11)  - 
Application. WorksheetFunction. Confidence (0 . 05,  Worksheets ( "Data" ) .Cells (2 , 
13),  (v  -  1)),  0) 

Worksheets ( "Data" ) .Cells (2 ,  16)  =  Worksheets ( "Data" ) .Cells (2 ,  11)  + 
Application. WorksheetFunction. Confidence (0 . 05,  Worksheets ( "Data" ) .Cells (2 , 
13)  ,  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (2 ,  15)  =  0 
Worksheets ( "Data" ) .Cells (2  ,  16)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (3 ,  13)  >  0  Then 
Worksheets ( "Data" ) .Cells (3 ,  15)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells (3,  11)  - 
Appl ication. WorksheetFunction. Confidence (0 .05,  Worksheets ( "Data" ) .Cells (3 , 
13),  (v  -  1)),  0) 

Worksheets ( "Data" ) .Cells (3 ,  16)  =  Worksheets ( "Data" ) .Cells (3 ,  11)  + 

Appl icat ion. WorksheetFunct ion. Conf idence (0 . 05 ,  Worksheets ( "Data" ) .Cells (3 , 
13)  ,  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (3 ,  15)  =  0 
Worksheets ( "Data" ) .Cells (3 ,  16)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (4,  13)  >  0  Then 
Worksheets ( "Data" ) .Cells  (4,  15)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells (4,  11)  - 
Application. WorksheetFunction. Conf idence (0.05,  Worksheets ( "Data" ) .Cells (4, 
13)  ,  (v  -  1)  )  ,  0) 

Worksheets ( "Data" ) .Cells (4,  16)  =  Worksheets ( "Data" ) .Cells (4,  11)  + 
Application. WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) .Cells (4, 
13),  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (4 ,  15)  =0 
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Worksheets ( "Data" ) .Cells (4 ,  16)  =  0 

End  If 

If  Worksheets ( "Data” ) .Cells (5,  13)  >  0  Then 
Worksheets ( "Data" ) .Cells (5,  15)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells (5,  11)  - 
Application . WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) . Cells (5 , 
13)  ,  (v  -  1) ) ,  0) 

Worksheets ( "Data" ) .Cells (5,  16)  =  Worksheets ( "Data" ) .Cells (5 ,  11)  + 
Application. WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) .Cells (5, 
13)  ,  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (5 ,  15)  =  0 
Worksheets ( "Data" ) .Cells (5,  16)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (6,  13)  >  0  Then 
Worksheets ( "Data" ) .Cells (6,  15)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells (6,  11)  - 
Application. WorksheetFunction. Confidence (0 . 05 ,  Worksheets ( "Data" ) .Cells (6 , 
13),  (v  -  1)),  0) 

Worksheets ( "Data” ) .Cells (6,  16)  =  Worksheets ( "Data" ) .Cells (6,  11)  + 
Applicat ion. WorksheetFunction. Confidence (0 . 05 ,  Worksheets ( "Data" ) .Cells (6 , 
13),  v  -  1) 

Else 

Worksheets ( "Data” ) .Cells (6 ,  15)  =  0 
Worksheets ( "Data" ) .Cells (6 ,  16)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (7 ,  13)  >0  Then 
Worksheets ( "Data" ) .Cells  (7 ,  15)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells(7,  11)  - 
Application. WorksheetFunction. Confidence (0 . 05,  Worksheets ( "Data" ) .Cells (7 , 
13)  ,  (v  -  1)  )  ,  0) 

Worksheets ( "Data" ) .Cells (7 ,  16)  =  Worksheets ( "Data" ) .Cells (7 ,  11)  + 
Applicat ion. WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) .Cells (7 , 
13),  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (7,  15)  =  0 
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Worksheets ( "Data" ) .Cells (7 ,  16)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (8,  13)  >  0  Then 
Worksheets ( "Data" ) .Cells (8 ,  15)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data” ) .Cells(8,  11)  - 
Application. WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) .Cells (8, 
13),  (v  -  1)),  0) 

Worksheets ( "Data” ) .Cells (8,  16)  =  Worksheets ( "Data" ). Cells (8 ,  11)  + 
Application. WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) .Cells (8 , 
13)  ,  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (8,  15)  =  0 
Worksheets ( "Data" ) .Cells (8,  16)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (9 ,  13)  >  0  Then 
Worksheets ( "Data" ) .Cells (9,  15)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells (9,  11)  - 
Application. WorksheetFunction. Confidence (0 . 05 ,  Worksheets ( "Data" ) .Cells (9 , 
13)  ,  (v  -  1)) ,  0) 

Worksheets ( "Data" ) .Cells  (9 ,  16)  =  Worksheets ( "Data" ) .Cells  (9 ,  11)  + 
Application .WorksheetFunction .Confidence (0.05,  Worksheets ( "Data" ) .Cells (9 , 
13)  ,  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (9 ,  15)  =  0 
Worksheets ( "Data" ) .Cells (9 ,  16)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (10,  13)  >  0  Then 
Worksheets ( "Data" ) .Cells  (10 ,  15)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells(10,  11)  - 
Application.WorksheetFunction.Conf idence (0 . 05 ,  Worksheets ( "Data" ) .Cells (10 , 
13)  ,  (v  -  1) ) ,  0) 

Worksheets ( "Data" ) .Cells (10 ,  16)  =  Worksheets ( "Data" ) .Cells (10 ,  11)  + 
Application.WorksheetFunction.Conf idence (0 .05,  Worksheets ( "Data" ) .Cells (10 , 
13)  ,  v  -  1) 

Else 
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Worksheets ( "Data" ) .Cells (10,  15)  =  0 
Worksheets ( "Data" ) .Cells (10,  16)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (11 ,  13)  >  0  Then 
Worksheets ( "Data" ) .Cells (11 ,  15)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells (11,  11)  - 
Application.WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) .Cells (11 , 
13)  ,  (v  -  1)  )  ,  0) 

Worksheets ( "Data" ) .Cells (11,  16)  =  Worksheets ( "Data" ) .Cells (11,  11)  + 
Application. WorksheetFunction. Conf idence (0 .05,  Worksheets ( "Data" ) .Cells (11, 
13)  ,  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (11,  15)  =  0 
Worksheets ( "Data" ) .Cells (11,  16)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (12 ,  13)  >  0  Then 
Worksheets ( "Data" ) .Cells (12 ,  15)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells(12,  11)  - 
Application. WorksheetFunction. Confidence (0 . 05,  Worksheets ( "Data" ) .Cells (12 , 
13),  (v  -  1)),  0) 

Worksheets ( "Data" ). Cells (12 ,  16)  =  Worksheets ( "Data" ) .Cells (12 ,  11)  + 
Application. WorksheetFunction. Conf idence (0.05,  Worksheets ( "Data" ) .Cells (12 , 
13),  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (12 ,  15)  =  0 
Worksheets ( "Data" ) .Cells (12 ,  16)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (13 ,  13)  >  0  Then 
Worksheets ( "Data" ) .Cells (13 ,  15)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells(13,  11)  - 
Application. WorksheetFunction. Conf idence (0.05,  Worksheets ( "Data" ) .Cells (13 , 
13),  (v  -  1)),  0) 

Worksheets ( "Data" ) .Cells (13 ,  16)  =  Worksheets ( "Data" ) .Cells (13 ,  11)  + 
Application. WorksheetFunc tion. Conf idence (0.05,  Worksheets ( "Data" ) .Cells  (13 , 
13)  ,  v  -  1) 
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Else 


Worksheets ( "Data" ) .Cells (13  ,  15)  =  0 
Worksheets ( "Data" ) .Cells (13 ,  16)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (14,  13)  >  0  Then 
Worksheets ( "Data" ) .Cells (14,  15)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells(14,  11)  - 
Application. WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) .Cells (14, 
13) ,  (v  -  1)  )  ,  0) 

Worksheets ( "Data" ) .Cells (14 ,  16)  =  Worksheets ( "Data" ) .Cells (14,  11)  + 
Application. WorksheetFunction. Confidence (0 . 05 ,  Worksheets ( "Data" ) .Cells  (14 , 
13)  ,  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (14 ,  15)  =  0 
Worksheets ("Data") .Cells (14,  16)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (15 ,  13)  >  0  Then 
Worksheets ( "Data" ) .Cells (15 ,  15)  = 

Application. WorksheetFunction. Max (Worksheets ( "Data" ) .Cells (15,  11)  - 
Application. WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) . Cells (15 , 
13),  (v  -  1)),  0) 

Worksheets ( "Data" ) .Cells (15 ,  16)  =  Worksheets ( "Data" ) .Cells (15,  11)  + 
Appl ication. WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) .Cells (15 , 
13)  ,  v  -  1) 

Else 

Worksheets ( "Data" ) .Cells (15 ,  15)  =  0 
Worksheets ( "Data" ) .Cells (15 ,  16)  =  0 

End  If 

If  Worksheets ( "Data" ) .Cells (16,  13)  >  0  Then 
Worksheets ( "Data" ) .Cells (16,  15)  = 

Application. WorksheetFunction. Max (Worksheets { "Data" ) .Cells (16,  11)  - 
Appl ication. WorksheetFunction. Confidence (0.05,  Worksheets ( "Data" ) .Cells (16, 
13),  (v  -  1)),  0) 
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Worksheets ( "Data” ) .Cells (16,  16)  =  Worksheets ( "Data" ) .Cells (16,  11)  + 
Application.WorksheetFunction.Confiden.ee (0.05,  Worksheets ( "Data" ) .Cells (16, 
13)  ,  v  -  1) 

Else 

Worksheets { "Data” ) .Cells (16,  15)  =  0 
Worksheets ( "Data” ) .Cells (16,  16)  =  0 

End  Sub 
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EXHIBIT  2  APPENDIX  E  STATISTICAL  RESULTS  FOR  VARYING  Pc 


All  table  listed  below  were  computed  using  1,000  replications  of  the  simulation. 

Table  A1  displays  statistical  summaries  of  the  output  from  simulation  of  the  number  of  mission  aborts 
occurring  as  a  result  of  1,250  missions  (5,000  flight  hours)  while  varying  Pc.  The  parameters  of  the  beta 
distribution  are  as  per  Table  1 1;  the  parameters  of  the  Weibull  distribution  are  aw=l-5  and  Pw=1108.  This  is 
data  for  Figure  6. 


Table  A1 


Pc 

LEGACY  PROCESS 

CBM+  PROCESS 

0.10 

KMWSfi'El 

IBEXES 

4.70 

SD: 

2.14 

4.83 

Mean: 

5.25 

SD 

2  27 

539 

0.20 

UMiW 

Mean: 

4.53 

SD: 

2.11 

■iBiiMWa 

4.66 

Mean: 

2.95 

SD 

1.64 

■Baa 

3.06 

0.30 

MiMiM 

Mean: 

4.57 

SD: 

2.07 

■iliiiiMP 

4.70 

Mean: 

1.91 

SD 

1.40 

i^EEEH 

2.00 

0.40 

UMiW 

Mean: 

4.49 

SD: 

2.09 

4.62 

Mean: 

1.32 

SD 

1.11 

1.39 

0.50 

HWiliW 

Mean: 

4.45 

SD- 

2.15 

IKE9H 

4.58 

Mean: 

0.93 

SD 

0.97 

0.99 

0.60 

I^B^EB 

4.55 

SD: 

2.13 

■iMliMB 

kzek 

4.68 

0.73 

SD 

0.87 

0.78 

0.70 

MJliMlM 

■EISES 

4.56 

SD 

2.06 

■■■ 

0.53 

SD 

0.74 

0.57 

0.80 

HJM1M 

^■E?EEB 

4.52 

SD: 

2.08 

keeh 

4.65 

Mean: 

0.46 

SD 

0.68 

0.50 

0.90 

fcyjsisiSCT 

4.51 

SD 

2.08 

■3.-ii«sni 

■I.M 

^■EBB 

0.36 

SD 

■n 

MiKVKlH 

0.40 

0.99 

■DBS 

4.53 

SD: 

2.14 

■EEQH 

4.66 

■DBS 

0.32 

SD 

0.58 

0.35 

Table  A2  displays  the  statistical  summaries  of  the  output  from  simulation  of  the  repair  times  occurring  as  a 
result  of  1,250  missions  (5,000  flight  hours)  while  varying  Pc.  The  parameters  of  the  beta  distribution  are  as 
per  Table  1 1;  the  parameters  of  the  Weibull  distribution  are  aw=1.5  and  pw=  1 1.08.  This  is  data  for  Figure  7. 

Table  A2 


Pc 

LEGACY  PROCESS 

CBM+  PROCESS 

0.10 

IR 

I^DSSS 

■MW 

SD: 

■tHktsM 

■MM 

IR 

IBEXES 

KQQB 

SD 

1M 

■4:1 ■ 

0.20 

IR 

usskei 

■MiW 

SD: 

■dsM 

■3.-KMB 

■«W 

■r.*4w 

IR 

■HHli 

SD 

■*1 :>■ 

■ifcliliW 

0.30 

IR 

USHEMi 

W4-THW 

SD: 

»W 

■i1A««H 

IR 

I^DESfiS 

SD 

■*W 

■:»««!  B 

■sMMiM 

0.40 

IR 

ISSES 

■f»-W 

SD: 

■>>41  kM 

■i1A««H 

muetm 

IR 

i^BEm 

SD 

HKBW 

■:!A««ia 

■iliikW 

0.50 

IR 

■MW 

SD: 

■>i*M 

■:1iV-T«H 

■MM 

IR 

Mean: 

■dlEM 

SD 

■=1AHria 

nw 

■iWHsM 

0.60 

IR 

waho-w 

SD: 

■iW/MB 

IR 

Mean: 

SD 

18.23 

W.L1.1W 

47.81 

0.70 

IR 

Mean: 

■MM 

SD: 

■&V/MB 

IR 

Mean: 

SD 

17.29 

■;!A««ia 

W.H-L-W 

45.14 

0.80 

IR 

ji^BESI 

■MM 

SD: 

■MIM 

■19 

77.58 

IR 

Mean: 

WIitAM 

SD 

16.05 

WiHrMH 

wroaw 

41.74 

0.90 

IR 

I1BESII1 

■aw 

SD: 

■MW 

WS.V4TO8 

■MM 

IR 

Mean: 

■iHW 

SD 

15.80 

waamin 

wcfiiaiw 

wnaenw 

IR 

EHEH5! 

SD: 

■MBiM 

WSHiMWB 

»*»  w 

UBEM 

IR 

Mean: 

SD 

14.53 

wtkisitw 

Table  A3  displays  the  statistical  summaries  of  the  simulation  output  for  the  number  of  diagnostic  symptoms 
recognized,  number  of  mission  aborts,  repair  time  as  a  result  of  1,250  missions  (5,000  flight  hours)  while 
varying  the  variance  of  the  Weibull  distribution  determining  MA  for  the  case  with  mean  Pc=0.99.  The  beta 
distribution  has  parameters  ae=25  and  Pb=.25.  The  parameters  of  the  Weibull  distribution  when  aw=0.5  then 
Pw=5,  when  aw=l  then  pw=10,  and  when  aw=1.5  then  pw=l  1.08.  This  is  data  for  Figures  8,  9,  and  10. 
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Table  A3 


a 

LEGACY  PROCESS 

CBM+ PROCESS 

0  50 

msm 

SD: 

1.98 

Mean: 

7.26 

SD 

2.64 

7.42 

0.50 

— SEEiSP 

7.35 

SD: 

2.76 

■iiiilMrtB 

7.52 

KB 

Mean: 

4.17 

SD 

2.07 

4.30 

0.50 

XR 

HJH 

SD: 

i  mer-x-zm 

71.78 

XR 

Mean: 

»U;f.  ■ 

SD 

18.51 

■iiAMtiB 

EES 

1.00 

IDSs 

SD: 

2.48 

6.23 

XDSs 

Mean: 

SD 

3.28 

eh 

1.00 

KgHif.TiTl 

mSSB 

5.44 

SD: 

2.25 

HIEUE 

5.58 

Mean: 

1.12 

SD 

1.03 

HK2H 

1.18 

1.00 

XR 

SD: 

essi 

XR 

Mean: 

SD 

16.35 

■ifefciMB 

1.50 

6.79 

SD: 

2.66 

■iiiiiMtiB 

HsE£H 

6.95 

Mean: 

■IliM 

SD 

3.36 

IP1HIM 

11.24 

1.50 

i— aimfi’S 

4.52 

SD: 

2.12 

■:!AM«!B 

4.65 

Mean: 

his^h 

SD 

0.51 

eeb 

0.31 

1.50 

XR 

74.81 

SD: 

■slAMtlB 

EH 

XR 

Mean: 

SD 

14.80 

35.70 

Tables  A4  and  A5  display  the  statistical  summaries  for  the  simulation  output  of  the  number  of  diagnostic 
symptoms  recognized,  number  of  mission  aborts,  repair  time  as  a  result  of  1,250  missions  (5,000  flight  hours) 
while  varying  the  variance  of  the  Weibull  distribution  having  mean  10  determining  MA  for  the  cases  where 
mean  Pc=0.50  and  mean  Pc=0.20;  the  parameters  of  the  beta  distribution  are  cib=25  pB=25  (when  Pc=0.50)  and 
(Xb=25  pB=100  (when  Pc=0.20).  This  is  data  for  Figures  8,  9,  and  10. 


Table  A4  (Pc=0.50) 


a 

LEGACY  PROCESS 

CBM+  PROCESS 

0.5 

BBDBiS 

4.19 

SD 

2.02 

HEE^H 

4.31 

eh 

i^BZMBB 

6.39 

SD: 

2.54 

MC.SV 

0.5 

MJiWiM 

■BiiSO 

7.32 

SD 

2.70 

■iBSMB 

7.49 

MBM5ESSB 

5.12 

SD: 

2.28 

eelh 

0.5 

XR 

SD 

\Wrl&im 

XR 

SD: 

■:!AM«!B 

ebb 

1 

EH 

■■S5H 

5.99 

SD 

2.44 

tMMI 

6.14 

EH 

SD: 

3.12 

■fttiMB 

H3£H 

9.49 

1 

I^BBEB 

SD 

2.34 

BEB 

5.42 

H9 

SD: 

1.45 

2.05 

1 

XR 

SD 

28.77 

Msaa 

rnanu 

mimsm 

XR 

■BB 

■.iW 

SD: 

naa 

■mess 

■aaeM 

1.5 

EJ»1  1 M 

{■EBB 

SD 

2.50 

■s&tfl 

6.87 

slices 

I^BiBBB 

eeeb 

SD: 

3.10 

■mi  M 

meem 

1.5 

Emma 

BWHET'4: 

SD 

2.10 

■sMMMB 

4.75 

EH 

SD: 

0.98 

■iliBWB 

1.06 

1.5 

ZB 

[■EBB 

SD 

■iliiiMiiB 

mkMtm 

XR 

^ IB 

■Ji W 

SD: 

EIsliIsM 

■aw 

Table  A5  (Pc=0.20) 


a 

LEGACY  PROCESS 

CBM+  PROCESS 

0.5 

El»l  1  ■ 

f^BBBB 

gHKUH 

SD 

1.98 

^eezh 

4.18 

eh 

j^BSISSB 

4.88 

SD: 

2.27 

■inmna 

ebi 

5.02 

0.5 

■yj«T*Ti£B 

i^bbbb 

7.26 

SD 

2.62 

■:M7T»!B 

ebh 

7.42 

MPXcvyI 

6.44 

SD: 

2.45 

MiliKWB 

eese 

6.59 

0.5 

XR 

Mean: 

■MM 

SD 

»liliM«IB 

■r«;« 

XR 

SD: 

■iW/MB 

■gL*LE 

1 

Mean: 

SD 

2.47 

6.25 

eh 

ihesb 

7.56 

SD: 

2.72 

MiliKWB 

7.72 

1 

■yj.TT.n 

SD 

2.39 

■iWMWB 

eeh 

5.54 

■yjgTigi 

I^HEBB 

3.94 

SD: 

2.09 

he 

4.07 

1 

XR 

Mean: 

SD 

l«s*« 

»MM«iB 

ESS 

XR 

SSHKS31 

SD: 

MiliKWB 

■WHIM 

1.5 

Mean: 

eeez^i 

SD 

2.59 

h^ej-h 

7.20 

eh 

8.67 

SD: 

2.95 

MiliKWB 

8.85 

1.5 

XJ.T.IiU 

i^BEEiSB 

4.54 

SD 

2.19 

eesh 

4.68 

EQsESE 

BKM 

2.91 

SD: 

1.72 

Miiaiitu 

3.01 

1.5 

XR 

Mean: 

SD 

27.79 

»!iaami 

EH 

XR 

SD: 

mssmum 

MiHHM 

Table  A6  displays  the  statistical  summaries  of  simulation  output  for  the  number  of  false  positives  and  the 
repair  (inspection)  times  as  a  result  of  1,250  missions  (5,000  flight  hours)  while  varying  the  false  positive 
arrival  rate;  the  parameters  of  the  beta  distribution  are  <xb=25  and  Pb=.25;  the  parameters  of  the  Weibull 
distribution  are  aw=l  -5  and  Pw=l  1.08.  This  is  an  expanded  version  of  Table  12. 

Table  A6 
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False 

Positive 

Rate 

CBM+  PROCESS  (NUMBER  OF  FALSE  POSITIVE  ARRIVALS) 

0.9999 

IFP 

Mean: 

1238.57 

SD: 

3.31 

95%  Cl: 

1238.36 

1238.78 

0.90 

IFP 

Mean: 

1115.29 

SD: 

11.18 

95%  Cl: 

1114.59 

1115.98 

0.80 

IFP 

Mean: 

990.51 

SD: 

14.47 

95%  Cl: 

989.61 

991.41 

0.70 

IFP 

Mean: 

866.70 

SD: 

16.58 

95%  Cl: 

865.67 

867.73 

0.60 

IFP 

Mean: 

743.41 

SD: 

17.13 

95%  Cl: 

742.34 

744.47 

0.50 

IFP 

Mean: 

620.23 

SD: 

17.69 

95%  Cl: 

619.14 

621.33 

0.40 

IFP 

Mean: 

495.29 

SD: 

16.85 

95%  Cl: 

494.25 

496.34 

0.30 

IFP 

Mean: 

372.07 

SD: 

16.24 

95%  Cl: 

371.06 

373.08 

0.20 

IFP 

Mean: 

247.05 

SD: 

14.09 

95%  Cl: 

246.18 

247.92 

0.10 

IFP 

Mean: 

124.34 

SD: 

10.44 

95%  Cl: 

123.69 

124.98 

0.0001 

IFP 

Mean: 

0.14 

SD: 

0.36 

95%  Cl: 

0.11 

0.16 

False 

Positive 

CBM+  PROCESS  (REPAIR  TIME  INCURRED  DUE  TO  FALSE  POSITIVE  ARRIVALS) 

Rate 

0.9999 

IFP  Time 

Mean: 

3717.41 

SD: 

105.16 

95%  Cl: 

3710.89 

3723.93 

0.90 

IFP  Time 

Mean: 

3343.55 

SD: 

106.54 

95%  Cl: 

3336.95 

3350.15 

0.80 

IFP  Time 

Mean: 

2969.66 

SD: 

104.41 

95%  Cl: 

2963.19 

2976.13 

0.70 

IFP  Time 

Mean: 

2598.47 

SD: 

104.49 

95%  Cl: 

2591.99 

2604.94 

0.60 

IFP  Time 

Mean: 

2228.17 

SD: 

98.01 

95%  Cl: 

2222.10 

2234.25 

0.50 

IFP  Time 

Mean: 

1861.03 

SD: 

89.15 

95%  Cl: 

1855.50 

1866.55 

0.40 

IFP  Time 

Mean: 

1490.28 

SD: 

85.40 

95%  Cl: 

1484.98 

1495.57 

0.30 

Mean: 

1118.76 

SD: 

74.55 

95%  Cl: 

1114.14 

1123.38 

0.20 

Mean: 

741.94 

SD: 

62.68 

95%  Cl: 

738.05 

745.82 

0.10 

Mean: 

371.62 

SD: 

47.00 

95%  Cl: 

368.71 

374.54 

0.0001 

IFP  Time 

Mean: 

0.42 

SD: 

1.64 

95%  Cl: 

0.32 

0.52 

Table  A7  displays  the  statistical  summaries  of  the  output  of  simulations  for  the  total  repair  time  (total  repair 
time  =  diagnostic  symptom  repair  time  +  false  positive  repair  time)  as  a  result  of  1,250  missions  (5,000  flight 
hours)  while  varying  the  false  positive  arrival  rate.  The  parameters  of  the  beta  distribution  are  aB= 25  and 
Pb=25\  the  parameters  of  the  Weibull  distribution  are  ar*=1.5  and  /hf=l  1.08.  This  is  an  expanded  version  of 
Table  14. 


Table  A7 


False 

Positive 

Rate 

CBM+  PROCESS  (TOTAL  REPAIR  TIME=DS  REPAIR  TIME+FALSE  POSITIVE  REPAIR  TIME) 

0.10 

IR+IFP  Time 

Mean: 

406.13 

SD: 

48.87 

95%  Cl: 

403.11 

409.16 

0.09 

IR+IFPTime 

Mean: 

369.59 

SD: 

46.37 

95%  Cl: 

366.71 

372.46 

0.08 

IR+IFP  Time 

Mean: 

330.84 

SD: 

42.89 

95%  Cl: 

328.18 

333.50 

0.07 

IR+IFPTime 

Mean: 

291.50 

SD: 

42.19 

95%  Cl: 

288.89 

294.12 

0.06 

IR+IFPTime 

Mean: 

257.54 

SD: 

38.12 

95%  Cl: 

255.18 

259.91 

0.05 

IR+IFPTime 

Mean: 

221.93 

SD: 

37.29 

95%  Cl: 

219.61 

224.24 

0.04 

IR+IFPTime 

Mean: 

185.08 

SD: 

32.49 

95%  Cl: 

183.06 

187.09 

0.03 

IR+IFPTime 

Mean: 

146.10 

SD: 

30.02 

95%  Cl: 

144.24 

147.96 

0.02 

IR+IFPTime 

Mean: 

108.97 

SD: 

25.81 

95%  Cl: 

107.38 

110.57 

0.01 

IR+IFPTime 

Mean: 

71.91 

SD: 

20.47 

95%  Cl: 

70.64 

73.17 

0.0001 

IR+IFP  Time 

Mean: 

36.06 

SD: 

15.43 

95%  Cl: 

35.10 

37.02 

Table  A8  displays  the  statistical  summaries  of  simulation  output  for  the  time  awaiting  replacement  components 
as  a  result  of  1,250  missions  (5,000  flight  hours)  while  varying  Poh  when  mean  Pc  is  0.99  and  the  mean  time 
until  the  replacement  component  arrives  is  3  days.  The  parameters  of  the  beta  distribution  are  aB= 25  and 


120 


[>b=- 25;  the  parameters  of  the  Weibull  distribution  are  eur=  1.5  and  j$w=\  1 .08.  The  time  until  a  component 
replacement  arrives  has  an  exponential  distribution.  This  is  data  for  Figure  12. 

Table  A8 


P ON  HAND 

LEGACY  PROCESS 

CBM+  PROCESS 

IAS 

^EXUi^l 

SD: 

Ha 

0.01 

IAS 

Mean: 

^EXiH^I 

SD 

0.00 

9^9 

0.00 

0.90 

IAS 

JRftSESi 

1.72 

SD: 

3.23 

■iiiiiMtlB 

HE£9 

1.92 

IAS 

Mean: 

SD 

1.64 

9E9 

0.55 

0.80 

IAS 

K3U2M! 

3.57 

SD: 

4.77 

■iiiiiMtlB 

3.87 

IAS 

Mean: 

^EEZ^I 

SD 

2.27 

■iliUMB 

9iB9 

1.01 

0.70 

IAS 

SD: 

5.59 

^Q^9 

5.38 

IAS 

Mean: 

1.26 

SD 

2.83 

■sfcfciMB 

9^9 

1.44 

0.60 

IAS 

az&a 

SD: 

6.76 

■iiAMrtB 

7.47 

IAS 

Mean: 

1.83 

SD 

3.27 

HE£9 

2.03 

0.50 

IAS 

IHESSl 

7.99 

SD: 

7.05 

■K1AMWB 

9Z2S9 

8.43 

IAS 

Mean: 

2.09 

SD 

3.40 

2.30 

0.40 

IAS 

Bl'lfl’l 

SD: 

8.02 

■iiiiliMB 

■HUB 

IAS 

Mean: 

3.01 

SD 

4.19 

9ES29I 

3.27 

0.30 

IAS 

biihb 

SD: 

8.58 

■ilAiMB 

■U9 

IAS 

Mean: 

3.15 

SD 

4.37 

9933 1 

3.42 

0.20 

IAS 

Mean: 

HFfr'M 

SD: 

9.05 

■iMMMB 

9hTH9 

IAS 

Mean: 

3.36 

SD 

4.12 

HXH9 

3.61 

0.10 

IAS 

Mean: 

BUtB 

SD: 

9.47 

■ilAMdB 

9H1‘9 

■MilB 

IAS 

Mean: 

4.23 

SD 

5.65 

4.58 

■min™ 

^^9 

■BB 

bebm 

SD: 

mw 

9fnr9 

17.40 

^^9 

^9S^SS 

4.72 

SD 

5.57 

■sMiMtia 

KE2H 

5.06 

Table  A9  displays  the  statistical  summaries  of  simulation  output  for  the  time  awaiting  replacement  components 
as  a  result  of  1,250  missions  (5,000  flight  hours)  while  varying  Poh  when  mean  Pc  is  0.50  and  the  mean  time 
until  the  replacement  component  arrives  is  3  days.  The  parameters  of  the  beta  distribution  are  og=25  and 
/l/;=25;  the  parameters  of  the  Weibull  distribution  are  aw=  1.5  and  /Ar—  1 1.08.  The  time  until  a  replacement 
component  arrives  has  an  exponential  distribution.  This  is  data  for  Figure  13. 

Table  A9 


.  . HIM— 

CBM+  PROCESS 

Mean: 

^EXIi^B 

so 

0.01 

0.00 

HjSH9 

SD 

0.00 

9ES9 

0.00 

0.90 

919 

SD: 

3.47 

aiiHMB 

9E£9 

1.98 

■ta 

Bangui 

929 

SD 

1.86 

■sfcHMB 

9E9 

0.81 

0.80 

KB 

9219 

SD. 

4.53 

aiiHMB 

9XE9 

3.69 

■a 

1.38 

SD 

2.80 

■sfcKMB 

9B9 

1.56 

0.70 

■H 

H9 

SD. 

5.55 

aiiHMB 

^929 

■SEH 

2.26 

SD 

3.80 

■siiKMa 

|^B9 

2.49 

0.60 

K9I 

9SH9 

SD. 

6.32 

aiiHMB 

9X£9 

7.27 

BBS333I 

^ ■ 

SD 

3.86 

■sfcKMB 

i—J r.»;rei 

2.94 

0.50 

na 

9££9 

SD' 

6.70 

9£29 

9.10 

na 

3.43 

SD 

4.55 

■iiitM«ia 

919 

3.71 

0.40 

^a 

HHHSl! 

9I‘TM9 

SD: 

8.02 

aiiHMB 

9^9 

HBS1BS53 

3.92 

SD 

4.66 

■sfcKMa 

9E*9 

4.21 

0.30 

Ha 

Ibeebe 

SD. 

8.35 

aiiHMB 

mmm 

imi 

na 

5.11 

SD 

■iTiKMa 

hob 

5.46 

0.20 

Ka 

Mean: 

■MM 

SD: 

9.00 

»:%-VT«!B 

9KI39 

■ea»i»a 

«« 

HEMS! 

SD 

5.67 

^39 

5.85 

0.10 

■ta 

Mean: 

9HH9 

SD: 

9.27 

■ililiiMB 

9EM 

ana 

wa 

6.30 

SD 

6.12 

■sTiKMa 

932E9 

6.68 

■min™ 

■ta 

Mean: 

9MW 

SD: 

■sMflMB 

■IsIsfcM 

hmhm 

■fljjiUM 

6.77 

SD 

6.14 

■iTiKMa 

9sE£9 

7.15 

Table  A10  displays  statistical  summaries  of  simulation  output  for  the  time  awaiting  replacement  components 
as  a  result  of  1,250  missions  (5,000  flight  hours)  while  varying  Poh  when  mean  Pc  is  0.20  and  the  mean  time 
until  the  replacement  component  arrives  is  3  days.  The  parameters  of  the  beta  distribution  are  aB=25  and 
[:>b=  100;  the  parameters  of  the  Weibull  distribution  are  aw=\  .5  and  j>w=\  1  08.  The  time  until  a  replacement 
component  arrives  has  an  exponential  distribution.  This  is  data  for  Figure  14. 

Table  A 10 
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LEGACY  PROCESS 

IAS 

SD: 

0.00 

K&9 

■EH 

IAS 

MHAEslii 

SD 

0.00 

■EH 

0.00 

0.90 

IAS 

1.69 

SD: 

3.10 

1.88 

IAS 

■Si 

1.32 

SD 

2.91 

1.50 

0.80 

IAS 

3.34 

SD: 

4.51 

MiliiflWB 

H&MH 

3.62 

IAS 

IHMSi 

2.67 

SD 

4.05 

2.92 

0.70 

IAS 

SD: 

5.17 

5.38 

IAS 

Mean: 

3.81 

SD 

4.92 

4.12 

0.60 

IAS 

6.37 

SD: 

6.04 

■iiAMtlE 

6.75 

IAS 

Mean: 

5.25 

SD 

5.43 

5.58 

0.50 

IAS 

hbssb 

8.78 

SD: 

7.08 

9.22 

IAS 

Mean: 

5.92 

SD 

5.74 

6.28 

0.40 

IAS 

HUSEEgH 

MiXf 

SD: 

8.04 

■iiiiiMtlB 

BliEiB 

IAS 

Mean: 

SD 

7.25 

musui 

8.48 

0.30 

IAS 

ES5EH 

SD: 

8.40 

■ilAMdB 

HW 

HWBgBWa 

IAS 

Mean: 

9.31 

SD 

7.65 

9.79 

0.20 

■Ull 

SD: 

9.34 

HhllH 

Tll'l 

HF£gHI 

BEEB 

SD 

7.37 

■^1 

10.53 

0.10 

IAS 

Mean: 

SD: 

9.55 

TTTf 

bi-ba 

IAS 

Mean: 

■nw 

SD 

8.33 

■l‘TTf 

11.73 

■min™ 

IAS 

Mean: 

BMW 

SD: 

IBSESB 

Hf-fTT 

17-81 

IAS 

Mean: 

SD 

8.69 

■K^l 

12.96 

Table  All  displays  the  statistical  summaries  of  simulation  output  for  the  time  awaiting  replacement 
components  as  a  result  of  1,250  missions  (5,000  flight  hours)  while  varying  Poh  when  Pc  is  0.99  and  the  mean 
time  until  the  replacement  component  arrives  is  10  days.  The  parameters  of  the  beta  distribution  are  aB= 25  and 
f]B=. 25;  the  parameters  of  the  Weibull  distribution  are  a.w=  1.5  and  ftiv=  11.08.  The  time  until  a  replacement 
component  arrives  has  an  exponential  distribution.  This  is  data  for  Figure  15. 

Table  All 


EH 

LEGACY  PROCESS 

CBM+  PROCESS 

■mUBBM 

SD: 

042 

BEEE^I 

0.01 

0.00 

SD 

ooo 

beh 

0.00 

0.90 

IMBEDS 

9.44 

SD: 

IBM 

KliliiMB 

bsstiss 

1  in wm 

Ka 

HSSB 

6.67 

SD 

BEB 

KtiliH 

7.36 

0.80 

MESS— 

Mean: 

■KM 

SD' 

KbfcliM 

KliKMB 

BI'HTB 

Ka 

(BBSS 

■HtM 

SD 

1  14.44 

KMMMB 

MBBiM 

0.70 

■ta 

Mean: 

SD 

KliKMB 

bei^eb 

wsa 

BSBlliSS 

PHH'M 

SD 

HfcfckM 

Kraaa 

0.60 

IBEEIiB 

SD: 

BMW 

KliKMB 

KMEMM 

pia 

BEISBB 

SD 

■fclAMriB 

m&EkM 

KMtta 

0.50 

wM 

Mean: 

SD: 

mourn 

■sMiiME 

■IBM 

mrr+tm 

wxa 

KJil 

SD 

■:!A««1B 

MtUM 

0.40 

na 

IIBBEIIB 

■E3KE1 

SD: 

■A1EM 

KliKMB 
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Table  A12  displays  the  statistical  summaries  of  simulation  output  for  the  time  awaiting  replacement 
components  as  a  result  of  1,250  missions  (5,000  flight  hours)  while  varying  Poh  when  Pc  is  0.50  and  the  mean 
time  until  the  replacement  component  arrives  is  10  days.  The  parameters  of  the  beta  distribution  are  aB= 25  and 
Pb= 25;  the  parameters  of  the  Weibull  distribution  are  a^l.5  and  /A— I  1 .08.  The  time  until  a  replacement 
component  arrives  has  an  exponential  distribution.  This  is  data  for  Figure  16. 
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Table  A 13  displays  the  statistical  summaries  of  simulation  output  for  the  time  awaiting  replacement 
components  as  a  result  of  1,250  missions  (5,000  flight  hours)  while  varying  POH  when  Pc  is  0.20  and  the  mean 
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time  until  the  replacement  component  arrives  is  10  days.  The  parameters  of  the  beta  distribution  are  aB=25  and 
100;  the  parameters  of  the  Weibull  distribution  are  a.u=\  .5  and  f>w=1  1 .08.  The  time  until  a  replacement 
component  arrives  has  an  exponential  distribution.  This  is  data  for  Figure  17. 
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Exhibit  3:  Driving  Toward  Greater  Mutual  Understanding 


Condition-Based  Maintenance  (CBM) 
For  U.S.  Army  Aviation: 

A  Component  Selection  Methodology 


INFORMS  Conference 
San  Francisco,  California 
14  November  2005 


MAJ  Ernest  Wong 

Department  of  Systems  Engineering 
United  States  Military  Academy 


\Types  of  Maintenance  Paradigms 


•  Run-to-Failure  Maintenance 

-  Corrective  Maintenance 

-  “Driveway  Diagnostics” 

-  Reactive 


•  Time-Phased  Maintenance 

Preventative  Maintenance 

-  Calendar-Based 

-  Proactive 


•  Condition-Based  Maintenance 

-  Predictive  Prognostics 

-  Non-lnvasive 


Operations  Research  Center  of  Excellence 
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Soldiers 

Perform 

immediate 

Maintenance 

Actions 


Maintenance 
instructions  and 

supply  actions 
are  generated 


Data  loaded  into  an 
Enterprise  Data 
Warehouse 


Data  Warehouse 


Engineering  Analysis 

Is  Performed  to  Determine  Component  j 
Health  and  Remaining  Life 


The  CBM  Process 


Imbedded  Sensors 

1 .  Monitor  Component  Health 

2.  Data  Is  Downloaded  After 

Each  Flight 


Plane  Side  Work  Stations 

1.  Provide  Any  Immediate  Actions 

2.  Transfer  Data  Into  the  Management 
Information  System  (MIS) 


Unit  Maintenance  Production 
Control  Office 

1.  Optimizes  Maintenance  Activities 

2.  Transfers  Data  to  MIS 


Key  Enablers: 

Q  Imbedded  Sensors 
©  Data  Warehouse 
©  Engineering  Analysis 
©  Closed  Loop  Info  System 
©  High  Confidence  From  Users 


Logistics  Reg’s  | 
[Failure  Analysis 
I  Regime  Recog 


Army  Aviation’s  Take  on  CBM 


A  Set  of  Maintenance  Processes  and  Capabilities  That  Improve  Operational 
Availability  and  Reduce  the  Maintenance  Burden  on  the  Soldier  By: 

-  Enhancing  Diagnostics 

-  Evolving  to  Predicting  Remaining  Component  Life 

-  Culminating  into  Proactive  Supply  Transactions 

Derived  From  Near  Real-time  Assessment  &  Analysis  of  Data  From: 

-  Embedded  Sensors 

-  Platform  Maintenance  Environments 

-  Aircraft  and  Supply  Historical  Data 


•  To  the  Army  Leadership,  CBM  Is: 

-  A  Set  of  Proactive  Maintenance  Actions 

-  Based  on  Near-real  Situational  Awareness  (Health)  of  Equipment  Condition 

-  Obtained  From  Embedded  Sensors  and  Other  Sources 

-  Based  on  Evidence  of  Need  Rather  Than  Scheduled  Time  Periods 

To  Battlefield  Commanders,  CBM  Is: 

-  The  Ability  to  Meet  Mission  Requirements  With  Proactively  Driven  Maintenance 

-  The  Ability  to  Optimize  the  Competing  Demands  of  Warfighting  and  Planned  Maintenance 

To  the  Soldier,  CBM  Is: 

-  Maintenance  Instructions  Based  on  Actual  Condition  and  Usage 

-  Greatly  Enhanced  Diagnostics  and  Troubleshooting 

-  Reduced  Maintainer  Workload 

-  Reduced  or  Eliminated  Physical  Inspections  source  u  s  amcoa 
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Maintenance 

Intensive 

•  Inspections 


Reactive 

Unscheduled 


Army  Aviation’s  CBM  Vision 


Decrease  the  Maintenance  Burden  on  the  Soldier 
Increase  Platform  Availability  and  Readiness 
Reduce  Operations  &  Support  Costs 


•CBM  Proof  Of  Principle 

Feasibility  Demonstration 
•Selected  Components 
•Gradual  Extension  Between 
Inspections/Maintenance  Actions 
•Install  Diagnostic/Prognostics 
Equipment  on  Platforms 
•Develop  Enablers 


Maintenance 

Efficient 

■  Automated  Inspection 

-  Continual  Monitoring 

-  Reduced  Maintenance 
Inspections 

■  Predictive  &  Proactive 

■  Progressive  Improvement 
•  Fully  integrated  LCM 

■  Business  Process  Change 

■  Proactive  Supply 


Timeline  for  CBM  Implementation 

v.  .....  rv"Ti'  ■ 


Phase  1  -  Concept  Development  -  FY  05-07  u  ) 

-  Proof  of  Principle  -  Engineering  Analysis,  July  2005  *00^ 

-  Cross  Functional  Data  Warehouse  Development  for  Analysis 

-  Development  of  Diagnostics/Prognostic  Algorithms 

-  Assess  DOTMLPF  Impacts  &  Potential  Business  Process  Improvements 

-  Interoperability/Interfaces  With  GCSS-A,  CLOE,  PME,  LMP,  PLM+ 

Phase  2  -  Implementation  -  FY  08-15  i  iMlui 

-  Battalion  and/or  Brigade  Pilot  Implementation 

-  Modify  Business  Processes:  Forecasting,  Other  Supply  Chain  Initiatives 

-  Identify/Implement  Training/Doctrine/Publication  Changes 

-  Implementation  of  CBM  on  Aircraft  With  DSCs 

-  Examine  New  Technologies 

Phase  3  -  Operations  &  Process  Improvements  -  FY  16+ 

-  Sustain  CBM  Operations  ,  E 

-  Continuous  Analysis  of  Data 

-  Refinement  of  Prognostic  Capability 

-  Reassess  DOTLMLPF  and  Business  Processes 
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M  whks 

\X0w)  ^  f  . 


Realized  Benefits  of  CBM 


Aviation  Engineering  Directorate  Published  Airworthiness  Releases 
(as  of  16JUN05)  affecting  approximately  100  aircraft 


SYSTEM  ITEM 


AH-64  M/R 

Swashplate 


BENEFITS 


Eliminates  50  hr  bearing  inspection  (btw  1750  and  2250  hrs). 
Eliminates  maintenance  operational  check  (approximately  1  hr). 
MMH  saved  per  inspection:  7.4  hrs. 

Downtime  saved  per  aircraft:  5.9  hrs. 


Eliminates  vibration  checks  at  installation  and  phase. 
Extends  APU  mount  inspection  from  250  hrs  to  500  hrs. 
MMH  saved  per  inspection:  28.0  hrs. 

Downtime  saved  per  aircraft:  9.0  hrs. 


AH-64 

Aft  Hanger 
Bearing 

Safety  improvement  w/  continuous  diagnostic  monitoring. 
Extends  TBO  from  2500  hrs  to  2750  hrs.  ^  f 

MMH  saved  per  inspection:  4.4  hrs.  ^  |  | 

Fwd  Hanger 
Bearing 


Oil  Cooler  Axial 
Fan  Bearing 


Engine  Output 


Safety  improvement  w/  continuous  diagnostic  monitoring. 
Extends  TBO  from  2500  hrs  to  2750  hrs. 

MMH  saved  per  inspection:  4.4  hrs. 

Downtime  saved  per  aircraft:2.2  hrs. 


Eliminates  120  hr  inspection  w/  continuous  diagnostic  monitoring. 
Extends  TBO  from  2500  hrs  to  3000  hrs. 


Eliminates  AVA  installation  requirement  for  120  hr  inspection. 
Replace  w /  continuous  diagnostic  monitoring. 

MMH  saved  per  inspection:  3.3  hrs. 

Downtime  saved  per  aircraft:  1.8  hrs. 


Operations  Research  Center  of  Excellence 


Maintenance  Paradigm  Attributes 


1.  Component  contribution  to  vehicle’s  critical  functions.  How  will  this 


component's  degradation  impact  the  mission?  How  will  it  impact 
safety?  How  much  downtime  must  the  vehicle  endure  in  order  for 
the  component  to  be  repaired  or  replaced?  What  is  the  likelihood  of 
catastrophic  failure  associated  with  this  part? 

2.  Component  overall  cost.  This  is  more  than  just  the  simple  cost 


accounting  figure  for  the  component — also  included  are  costs 
associated  with  the  time  it  takes  to  replace  the  part,  shortage  costs, 
and  inventory  costs. 

3.  Complexity  of  the  component.  Will  it  be  feasible  to  determine  the-*#^ 


condition  of  a  particular  component?  Is  it  reasonable  to  expect  that 
this  part’s  condition  can  be  isolated  from  extraneous  noise? 

4.  Technological  capability  to  monitor  component  condition  (vibrations. 


temperature,  electronics,  etc.).  Is  there  a  history  of  monitoring  and 
assessing  the  particular  component  or  a  similar  component’s  status? 
Is  the  monitoring  likely  to  result  in  data  that  generates  enough 
fidelity  to  determine  component  degradation? 


Scientific  and 
Technological 
Advances  will 
Adjust 

s  P^sition^ — 


Easv  to  Monitor 


Bearings 
Drive  Shafts 


High  Need  to  * 
Monitor 
Component 


Component  Critltality 
Overall  Component  Cos 


potent  Complexity 
Technological  Capability 


Engines 

Hydraulic  Modules 


Dynamic  Components  Electronic  Units 


Operations  Research  Cen 


Relative  Ease  in 
Monitoring  Component 

Difficult 


Struts  &  Dampeners 
Pitch  Housings 
Ppatigue  Life -Limited  Parts 

-From  U.S.  AMCOM  Pamphlet  OCT05 

"  of  Excellence  • : •  •  1 6  of  1 §  : 
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C.D/rjpo/L&r,:  Co 


Scientific  and 
Technological 
Advances  will 
Adjust 

>  Positions'-— 


\  MP  Attribute  Relative  Importance 


High  Need  to 
Monitor 
Component 


Technological  Capability 


Hard  Relative  Ease  in 

Monitoring  Component 

•  Attribute  weighting  depends  largely  on  scientific  and  technological 
capabilities  that  enable  more  ropust  CBM  sensors  and  algorithms.  .  . 

•  Currently,  more  importance  placed  on  component  complexity  and  Hjj 
technological  capability. 

•  Shift  toward  greater  emphasis  on  component  criticality  and  overall  |  ll, 
component  cost  will  provide  a  more  top-down  systematic  approach.  ^ 


MP  Component  Categorization 


Maintenance  Paradigm  (MP)  Value 


0  Derived/Ancillary 
•  Platform/Core 
®  Breakthrough 
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Further  Research 


Developers  and  First  Users 

-  NASA 

-  DOD 


Mainstream  Adopters 

-  Auto  Manufacturers 

-  Commercial  Aviation 

-  Utilities  and  Power  Plants 

-  Hospitals  and  Medical  Device  Manufacturers 


New  Innovators 

-  Electronics 

-  Networks 

-  Materials 

-  Environment  &  Ecology 

-  Sports  Medicine 

-  NASCAR 


Operations  Research  Center  of  Excellence 
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Viability  of  This  Methodology 


¥ 


•  T.S.  Army  Aviation 

;  UH-60 

.  -  AH-64 
.  -  ch-47 


Future  Combat  System  (FCS) 


Joint  Strike  Fighter 
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Defining  Appropriate  Metrics  for 
Operating  Within  a  Condition-Based  Maintenance  Paradigm 


“The  victorious  army  first  realizes  the  conditions  for  victory,  and  then  seeks  to  engage  in  battle.  The  vanquished 
army  fights  first  and  then  seeks  victory.” 

--Sun  Tsu.  The  Art  of  War 


Executive  Summary 


Without  clear  objectives,  it  is  oftentimes  difficult  to  make  good  decisions.  To  come  up 
with  clear  objectives,  organizations  must  first  determine  what  is  important  and  then  figure  out  a 
way  to  assess  and  evaluate  how  well  they  are  able  to  perform  with  respect  to  what  is  important 
[7].  As  the  familiar  saying  goes,  “What  gets  measured,  gets  done.”  Metrics,  therefore,  play  a 
critical  role  in  organizational  performance,  and  leaders  today  are  very  concerned  about  exactly 
what  is  measured  [5],  Regardless  of  the  type,  size,  or  function  of  the  organization,  good  metrics 
help  to  drive  systemic  improvement  and  are  usually  [1]: 


•  Few  in  number  to  allow  concentration  on  those  vital  key  variables 

•  Linked  to  key  business  drivers  to  promote  organizational  success 

•  Composed  of  a  mix  of  past,  present,  and  future  to  achieve  a  holistic  perspective 

•  Based  around  needs  of  customers,  shareholders,  and  key  stakeholders 

•  Driven  from  the  top  and  permeated  throughout  all  levels  of  the  organization 

•  Composed  of  multiple  indices  to  give  a  better  overall  assessment 

•  Flexible  and  adaptable  to  changes  in  environment  and  strategy 

•  Based  on  targets  or  goals  established  through  research  rather  than  arbitrary  numbers 

Most  organizations  spend  countless  hours  collecting  and  interpreting  data  intended  to 
enhance  business  performance  and  productivity.  Yet  a  large  portion  of  this  time  amounts  to 
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nothing  more  than  wasted  effort  when  the  wrong  measurements  are  analyzed.  As  a  result, 
organizations  tend  to  create  unnecessary  procedures,  squander  resources,  and  detract  from  their 
objectives  when  this  happens.  Of  greater  consequence,  however,  is  that  wrong  metrics 
oftentimes  lead  organizations  to  inaccurate  decision-making  which,  in  turn,  translates  into  poor 
choices.  This  paper  examines  how  organizations  can  ensure  that  they  are  focused  on  a  set  of 
appropriate  metrics  that  are  linked  to  critical  success  factors,  such  as: 

•  Higher  quality  manufacturing  through  reduced  costly  mistakes 

•  Reduced  risk  and  improved  safety 

•  Increased  operational  efficiency  and  usage 

•  Extended  equipment  life  and  reduced  lifecycle  costs 

•  Increased  savings  in  personnel  training  costs 

•  Improved  employee  satisfaction 

•  Increased  confidence  in  major  decisions 

Developing  appropriate  metrics  to  evaluate  the  performance  of  the  condition-based 
maintenance  (CBM)  paradigm  for  U.S.  Army  aviation  is  critical  not  just  as  a  means  to  assess  the 
progress  of  the  program,  but  also  as  a  means  to  help  continually  improve  upon  the  CBM  vision 
of  achieving  optimal  operational  readiness  of  the  aviation  fleet.  DoD  Directive  5000.1 
articulates  some  of  the  critical  success  factors  for  CBM  [4]: 


•  Predict  equipment  failures  based  on  real-time  or  near  real-time  assessments  of 
equipment  condition  obtained  from  embedded  sensors 

•  Reduce  maintenance  down  time 

•  Increase  operational  readiness  by  repairing  or  replacing  system  components  based  on 
actual  condition  of  components  rather  than  on  a  scheduled  or  time-phased  basis 

Accordingly,  metrics  developed  for  CBM  must  relate  to  factors  such  as  these.  A  good  way  to 
help  ensure  that  the  metrics  are  meaningful  is  to  make  sure  the  chosen  measurements  are 
SMART— specific,  measurable,  aligned,  realistic,  and  time-bound  [6], 


Introduction — A  Shift  to  More  Meaningful  Metrics 


For  every  objective,  there  ought  to  be  at  least  one  quantifiable  metric  and  a  target  value 
for  that  metric.  Once  the  objective-level  target  for  a  metric  has  been  reached,  the  organization 
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knows  that  a  particular  objective  has  been  achieved.  Unfortunately,  most  organizations  try  to 
measure  too  many  things,  measure  the  wrong  things,  use  measures  that  no  longer  have  relevance, 
or  use  measures  that  are  ambiguous  and  give  managers  and  workers  little  control  [6],  When  this 
occurs,  a  great  deal  of  organizational  time,  energy,  and  resources  are  wasted. 

Michael  Dell,  CEO  of  Dell  Computers,  has  stated,  “To  motivate  an  employee  to  think 
like  an  owner,  you  have  to  give  her  metrics  she  can  embrace”  [3],  SMART  metrics — those  that 
are  specific,  measurable,  aligned  to  key  objectives,  realistic,  and  time-bound  (attainable  within  a 
certain  time  period) — are  a  key  element  of  a  performance  management  system  that  helps  link 
employees  to  the  success  of  the  organization  (Fig.  1). 
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Figure  1:  Aligning  Organizational  and  Individual  Goals  [6] 


However,  most  traditional  performance  management  systems  provide  “lagging”  rather 
than  “leading”  indicators.  They  tend  to  be  more  like  rearview  mirrors  that  explain  how  goals 
were  or  were  not  met  based  on  past  performance  rather  than  being  more  like  steering  wheels  that 
enable  organizations  to  adjust  to  changing  conditions  [5],  In  today’s  more  measure-based,  goals- 
driven,  performance  management  culture,  improved  metrics  must  be  developed  to  better  align 
efforts,  implement  strategies,  and  focus  on  results  (Fig.  2).  Organizations  that  make  a  concerted 
effort  to  shift  to  metrics  that  are  multidimensional,  permit  mid-course  steering,  create  value, 
display  line  of  sight  to  action,  enhance  strategy,  facilitate  management  across  multiple  functions, 
and  assist  in  the  management  of  output  value  are  much  more  likely  to  find  themselves  to  be  the 
leaders  in  this  new  century.  The  shift  to  more  meaningful  metrics  will  lead  to  the  advancement 
of  purposeful  organizations  that  succeed  at  effectively  and  efficiently  getting  things  done  right. 
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Figure  2:  A  Shift  to  More  Meaningful  Metrics  [5] 


A  Process  for  Linking  Meaningful  Metrics  to  Organizational  Objectives 


Dr.  Bob  Frost  has  developed  a  three-step  methodology  for  helping  develop  meaningful 
metrics  and  performance  indicators  [5]: 


•  Step  1 :  The  organization  examines  its  business  strategy  and  stakeholders  to  find 
crucial  Performance  Topics. 

•  Step  2:  The  organization  determines  where  and  how  it  must  succeed  on  each  topic, 
spelling  out  the  where’s  and  how’s  as  a  set  of  Critical  Success  Factors. 

•  Step  3:  The  organization  considers  each  Critical  Success  Factor  and  defines  specific 
Performance  Indicators  that  will  track  success  on  it. 


This  three-step  process  works  from  the  general  to  the  specific — from  Performance  Topic  to 
Critical  Success  Factors  to  specific  Performance  Indicators  (Fig.  3).  Taking  the  time  to  link 
performance  indicators  to  performance  topics  helps  to  ensure  the  chosen  metrics  will  be 
meaningful  and  provide  value  to  the  organization.  Raytheon  Corporation’s  Emery  Powell 
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Figure  3:  Sample  of  Frost’s  3-Step  Process  for  Developing  Meaningful  Performance  Indicators  [5] 

perhaps  summarizes  it  best  when  he  stated,  “A  strategy  without  metrics  is  just  a  wish.  And 
metrics  that  are  not  aligned  with  strategy  are  a  waste  of  time.” 

Meaningful  Performance  Metrics  for  CBM 

The  CBM  vision  of  achieving  optimal  operational  readiness  of  the  U.S.  Army’s  aviation 
fleet  already  has  a  set  of  clearly  articulated  Critical  Success  Factors.  According  to  DoD 
Directive  5000.1,  critical  to  the  success  of  the  CBM  paradigm  is  its  ability  to  [4]: 

•  Predict  equipment  failures  based  on  real-time  or  near  real-time  assessments  of 
equipment  condition  obtained  from  embedded  sensors 

•  Reduce  maintenance  down  time 

•  Increase  operational  readiness  by  repairing  or  replacing  system  components  based  on 
actual  condition  of  components  rather  than  on  a  scheduled  or  time-phased  basis 

As  such,  performance  indicators  and  metrics  developed  for  CBM  ought  to  align  with  Critical 
Success  Factors  such  as  these  (Fig.  4). 
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Figure  4:  A  Sample  3-Step  Metrics  Development  Chart  for  CBM 
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Figure  4  (cont.):  A  Sample  3-Step  Metrics  Development  Chart  for  CBM 


Conclusions — CBM  as  a  Viable  Strategy-Based  Performance  Management  System 

To  improve,  according  to  one  Webster’s  definition,  means  to  “enhance  in  value  or 
quality.”  This  definition  implies  that  value  and  quality  are  already  present.  The  CBM  paradigm 
aims  to  improve  upon  the  merits  of  the  more  widely-accepted  preventative  and  schedule-based 
maintenance  paradigm.  Today’s  enabling  technologies  permit  CBM  to  define  maintenance 
actions  based  on  actual  conditions  obtained  from  non-invasive  tests  and  predictive  prognostics 
rather  than  relying  on  specified  intervals  which  may  be  too  conservative — creating  unnecessary 
maintenance— or  too  aggressive — resulting  in  undue  risk.  Because  the  CBM  paradigm 
approaches  maintenance  in  a  more  objective  fashion,  it  follows  that  CBM  should  be  an  ideal 
strategy-based  performance  management  system  that  analytically  targets  continual  improvement 
(Fig.  5).  However,  success  for  CBM  depends  largely  on  the  performance  measures  chosen  to 
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Figure  5:  Strategy-Based  Performance  Management  Design  [5] 

gauge  the  program’s  progress.  Perhaps  more  importantly,  continual  improvement  within  CBM 
itself  depends  largely  on  the  development  and  selection  of  performance  indicators  that  are 
SMART — specific,  measurable,  aligned  to  key  objectives,  realistic,  and  time-bound. 


Jim  O’Brien,  University  of  Kentucky  Associate  Basketball  Coach,  once  remarked, 
“Excellence  is  the  unlimited  ability  to  improve  the  quality  of  what  you  have  to  offer.”  CBM  has 
the  potential  to  not  only  enhance,  but  also  revolutionize  the  way  the  Army  does  maintenance. 
Although  still  in  its  infant  stages,  CBM  is  already  demonstrating  viable  progress  in  predicting 
equipment  failure  based  on  actual  conditions,  reducing  unnecessary  maintenance,  and  increasing 
the  Army  aviation  fleet’s  warfighting  reliability  and  readiness.  With  the  appropriate  metrics, 
CBM  has  the  potential  to  facilitate  real-time  communications  that  is  required  of  highly  effective 
and  efficient  organizations  (Fig.  6).  With  the  right  metrics,  CBM  has  the  opportunity  to  align 
process  improvement  with  long-term  vision  in  order  to  make  our  Army  a  much  more  lethal 
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Figure  6:  Avoiding  Both  the  Bureaucratic  and  Chaos  Trap  [2] 


Figure  7:  Avoiding  Both  the  Overconnect  and  Disconnect  Trap  [2] 


fighting  force  (Fig.  7).  Ideal  for  this  new  century,  CBM  is  a  strategy-based  performance 
management  system  that  aims  to  achieve  organizational  excellence. 
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